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You’re a real company when... 


by Jason Cohen on April 20, 2007 


What marks the moment when you become a “real” 
company? 


Ever since I started Smart Bear in 2002, there’s been a 


running question: 
“When do you know you're a real company?” 
Which of these are true: You’re a real company when... 


e You have a domain name that isn’t weird. 

e Your business cards don’t say “VistaPrint” on the 
back. 

e You take PO’s in addition to credit cards, and you’ve 
gotten over the fact that, yes, you have to actually 
ship the product before they give you money. 

¢ You switch from Quicken to Quickbooks. 

¢ You hire the first real, full-time employee. 

¢ You pay someone else to print pay stubs and pay 
your monthly employment taxes because you're just 
tired of doing it yourself. 

e You have a sign on the building. 

e Areal artist does your website and handouts. 

e You have a 256-page color glossy hard-backed book 
chronicling your ascent in the world (Saw this at 
Adobe. Gorgeous. I asked everyone I met there about 


it and no one but the receptionist had ever seen it. 
It’s sitting in the lobby. The receptionist, by the way, 
knew exactly what their branch of Adobe (Ottawa) 
did and could articulate why people bought their 
products in about 15 seconds. Her elevator pitch was 
better than the 10 minute diatribe I got from a senior 
manager later that day.) 


I made one of those bold because that’s the one that’s al- 
ways stuck with me. We got the sign about a year ago, 


and it really was a proud moment. 
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Currently Smart Bear has everything but the glossy book 
(but we do have a book with a glossy cover with over 
7000 copies in circulation). The 8000’th user of our vari- 
ous software products just came on-line. That makes me 
[2009 Update: now 45,000 
books and 30,000 paying users] 


feel like a real company. 


That was written 16 years ago. Now in 2023, I have a 


new story. 


I needed a new one... it’s not hard to pay employees any- 
more, and you've probably never heard of VistaPrint, nor 


used Quicken. Well, it used to be funny: 
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~» Ben Radler says: 
“pep = January 19, 2010 at 03:09 Edit 


Your business cards don’t say “VistaPrint” on the back. 


This made me laugh out loud, for serious. 


_ David Cruz says: 
April 10, 2010 at 10:37 Edit 


“You have to switch from Quicken to Quickbooks.” 
That made me smile :) 


= David Cruz’s latest blog post: Soon, a PC will not have files, folders, 
or documents + =-. 


When I was a little boy growing up in Austin Texas, 
“Downtown” was this frightening place we never went 
to. Downtown had full of things like government build- 
ings and law firms and things that families didn’t go to, 


and there was crime! 


1 Austin was 1/10th its current size, and probably didn’t have 
that much crime, but to a little-boy brain, it was scary. 


But often we would come to the outskirts of Downtown 
to get bagels. My parents grew up in and around New 
York City, and they insisted there was only one place to 
get “real” bagels. So sometimes, on the weekend, we’d 
get in the station wagon, I would hop in my car seat, and 
wed go down to the bagel store on the edge of 
Downtown. Looking out window, I knew we were at the 
bagel store when I could see a long, sloping ramp that 
led a parking garage. I would wait in the car, staring at 
that scary office building across the street, they’d get the 


bagels, and then we’d immediately U-turn and get out of 


there. 


Fast forward 40 years, and I’m standing in a public park 
with our amazing CEO Heather Brunner and a few dozen 
of the many hundreds of people who worked at WP 
Engine, watching as our sign goes up on the building. We 
kept checking a picture of the company logo on our cell 
phones, ensuring that each little piece was correctly ori- 


ented. (We couldn’t remember ourselves!) 


That building, is the building with the sloping ramp, im- 
printed on my memory like a duckling on its mother, one 
of those inexplicable snatches of childhood memories. 


You're a real company when... 
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Discount gambit 


by Jason Cohen on March 22, 2008 


Discounting is the typical sales technique, but refusing to 
discount can lead to a much better business, even in the 
Enterprise. 


ANDERSON 


“No, it’s fine, I’ve just never seen 
coupons used in an acquisition.” 


Which of these pricing strategies is more persuasive? 


1. If you buy now, I’ll give you a discount. 
2. The price is going up, but if you buy now I will lock 


in your rate. 


Both are types of discount. The typical software sales 
strategy is #1. It’s often applied to get the customer to 
“close” before the end of the month or quarter or some 


other arbitrary time boundary. 


At first blush it seems harder to persuade with #2. After 
all, #1 means the customer pays less than #2, because 
#2 isn’t a real discount—it’s a discount against an imag- 


ined future price. 


But for me, the evidence is overwhelmingly in favor of 


#2. Here’s why, from the point of view of the customer. 


In strategy #1 there’s a discount if I “act now.” Hmm, so 
that means the “price” wasn’t really the price after all. 


The “price” must have included a nice slice of pure profit 


that apparently you’re willing to forego. So you were 
gouging me before? And the only reason I found out 


about it is that it happens to be the end of the quarter? 


Thus #1 breeds mistrust—the opposite of what you’re 
trying to establish with me, your customer. In #2 you’re 
looking out for my interests’ You're cluing me in that 
there might be a rate increase, and you're actively pro- 
tecting me from it. Sure, I know there may not be an in- 
crease, or it may not come for a while. But it’s still pro- 


tection, not a gouge that you graciously chose to reveal. 


1 This is an example of the “Love” type of WIP (Willingness-to- 
pay) 


Four years ago I was trialing .TEST from Parasoft. It was 
buggy; even after hours of remote desktop control with 
tech support we couldn’t get it to stay up long enough to 


scan my code. 


But the salesman was persistent. The conversation went 
like this, minus many minutes of sales-filler-language on 


his end of the phone: 


“How much will this cost me?” 

“$20,000.” 

“Wow, I thought you were going to say $2,000. That’s 
way out of my price range for one person and this 
product. In fact, I’ve looked at FxCop and NUnit and it 
seems to me I can do the same thing with free tools. I 
was willing to pay for some convenience, but not that 
much.” 

“Let me see what I can do.” 

“No nevermind, it’s, like, an order of magnitude 
problem.” 


He called back the next day. 
“$1,500.” 


I didn’t buy. I talked to someone who did, though. A ref- 
erence customer. That guy said he paid $20,000. I asked 


how he liked it and whether he encountered the crash 
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problems I was seeing. He said they hadn’t installed it 
yet, but the demo looked great. I made a mental note to 
try to understand the mentality and budget that forks out 
$20,000 for a nice demo. 


But getting back to the point. If he can go from $20,000 
to $1,500, maybe he will go to $1,000. Yes, this strategy 
means often you will extract extra money from me. But it 
also means I don’t know where the floor is, and I have 
every incentive to haggle. The process drags out, ending 
at gunpoint. Meanwhile your “customer relationship” 


is now more of a “hostage situation.” 


So let me get this straight: It’s better to get an extra 10% 
on every order, but create an adversarial environment 
with me, your cherished customer? This is enterprise 
sales, right, where the pilot is 30 seats and the roll-out is 
2,000? And you’re going to risk pissing me off over 10% 
on the 30-seat part? 


And now imagine if I had called back that reference cus- 
tomer and told him he could’ve had it for $1,500? Yet an- 
other problem with discounting—word gets around, 
meaningless differences in pricing is unfair, and now I, 


the customer, see you as plain old dishonest. 
Goodbye 2,000-seat order. 


Even if we set the honesty/relationship argument 


aside, there’s the matter of image. 


What kind of company provides a discount? Wal-Mart, 
Target, Walgreens. Try to get quotes for Microsoft or 
Oracle or IBM products for 1000 desktops. Everything’s 
negotiable, everything’s discountable. At best it conjures 
images of haggling and struggle; at worst of low-quality 
or the desperate need to “meet numbers” at the expense 


of everything else. 


Which companies don’t discount, ever? Apple, Google, 
Constant Contact. No discounts on iPhones. No haggling 
over AdWord prices. What’s the image? Desirable. The 
best. Worth paying for. The leader doesn’t have to com- 


promise. The leader isn’t desperate for orders. 


Strategy #2 implies growth. You've planted “higher 
prices” in my head now. “Supply” in software is unlimit- 
ed, so that must mean “demand” is increasing. The cus- 
tomer won’t go through that calculus, but it certainly 
feels like the product is becoming more valuable. 
Discounts feel like unloading unwanted product; price in- 


creases feel like success. 


Strategy #2 implies I’m part of a club. I’ve gotten in ear- 
ly, on the ground floor, before the product explodes in 
popularity and prices go up. And I’m rewarded for this 
support and loyalty with price protection. A “thank-you” 
from because I was part of it, because I was there before 
you were big and expensive, because I took that risk with 
you. That’s actually a good reason to give some a dis- 


count, and here you are, proactively giving it. 


So there it is. #1 means less money now, an adversarial 
relationship, a never-ending struggle over money, and a 
message that maybe the product needs a discount to be 
desirable. #2 means more money now, a consistent and 
fair pricing policy, an inclusive, special customer relation- 


ship, and a message of market leadership and growth. 


So why do 90% of software companies pick #1? 
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Distinguishing constructive criticism from bad business advice 


by Jason Cohen on December 29, 2008 


Beware of advice that tries to change who you are. True 
wisdom guides you to a better version of yourself. 


Haiku: 
With their eyes of ice 
high-powered executives 
know better than you. 


I was starry-eyed when Frank showed up 18 months after 
the birth of my company, Smart Bear. Picture it: Frank 
was a silver-haired ex-VP-of-Sales for a big, successful 
company. An IPO had made him comfortably wealthy, 
and now he wanted to hook up with a promising new 
startup. With his résumé and enough money that he nev- 


er had to work again, he could be choosy. 
And he chose little ol’ me! 


And boy did I (think I) needed him. “I’m just a software 
developer, I don’t know anything about sales” I explained 
to Gerry , who had already unknowingly cemented him- 
self as my mentor. “I’m just a lowly engineer, ignorant of 
the mystical voodoo of six-figure purchase orders. I don’t 


even play golf! Frank has been there.” 


Gerry tried to explain. “There’s no magic to it. I’ve sold 
companies for $100 million without stepping foot on a 
golf course. You have a good product, you have some 


customers, just keep going.” 


Of course I didn’t listen. 


So Frank and I headed down the road to partnership. 
Frank had lots of ideas that “we have to implement, oth- 
erwise no big company will give us money.” For example, 
we had to change our name. “Smart Bear” sounded to 
Frank like a dorky one-man shareware site. (Ahem...) We 
needed something stoic and corporate. His suggestion 
was “Software Test and Deployment Systems, Inc,” or 
STDS! for short. Big companies like acronyms, he ex- 
plained, as if “big company” is an entity that can ex- 


press an opinion. 


1 tt apparently didn’t occur to him that this acronym is already 
taken. At least we could have used viral marketing. 


There was more: We needed patents (“to keep competi- 
tors at bay”), we needed a sales team (“software doesn’t 
sell itself”), and we needed to lighten up (“TI like to knock 
off early on Fridays”). In fact it seemed I was doing very 


little right! 


Fortunately Gerry finally got through. “Let me get this 
straight,” he rightly admonished me for the fourth time, 
“this guy wants 50% of the company, he wants a salary 
while you get none, he’s going to work 35 hours a week 
while you work weekends, he’s not bringing any money 
to the company, and his big idea about how to get more 
customers is to change your name to venereal disease?” 


Gerry’s criticism of Frank was harsh—as harsh as Frank’s 
criticism of me—but Gerry was right and luckily I 
(finally!) heeded him. 


Two weeks later I landed a $50,000 deal with a “big 
company.” (It was Intuit.) A few years later we were 
doing millions in revenue with “big companies,” still 
called Smart Bear, still no patents, still no sales guys. 
Who knew? (Gerry knew.)? 


2 Editor’s Note: I later sold Smart Bear, and now it’s worth over 
$2,000,000,000. And still called Smart Bear. 
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You have to understand though, saying no to Frank was 
hard. He had the expertise, I didn’t. I was convinced that 
he was right and I was wrong. If it wasn’t for Gerry’s 


guidance, I would have gone through with it. 


So how do you separate the the good counsel from the 
bad—the Gerrys from the Franks? Both sounded like ad- 
vice and criticism, both were experienced, both were 


strong-willed, and both truly believed in what he was 


saying. 
Here are a few things that have helped me filter advice: 


e The best advice doesn’t come as a barrage of 
statements but rather from a series of questions, 
asked by someone playing devil’s advocate. Rarely 
will an advisor know more than you do about your 
domain of expertise but that doesn’t mean an outside 
voice is useless. Pointed questions force you to 
defend your choices. A healthy debate challenges 
your assumptions without implying they’re false. 
New ideas are batted around as a brainstorm rather 
than handed down as gospel. In my case, Frank 
presented his view as a series of statements, axioms 
even; Gerry couched his perspective as pointed 
questions that required either rebuttal or 
agreement.In fact, playing devil’s advocate is a great 
exercise to do periodically anyway. Find an 
intelligent foe, take her to lunch, and follow Scott 
Berkun’s advice about Rude O&A. 

e “This is how it’s done” is almost never a good reason. 
If the sole basis for the advice is “tradition,” it’s just 
momentum and you might be right to buck the trend. 
Example: Smart Bear posts its prices on the Internet 


rather than negotiating. Enterprise software sales are 
rarely done that way, but posting your price is honest 
and makes business sense, so we do it anyway. 

e It’s easy to cut down ideas; it’s hard to create and 
execute them. Give me any idea and I can find 
someone who thinks it’s dumb. So what? 
“Constructive” criticism means constructing, not just 
blasting. Look for advice with a clear method for 
implementation and a clear path for results. 

¢ Does the criticizer volunteer a way to measure the 
success of her new idea? If so, the only reason not to 
try it is time or money. This is the guiding principle 
behind our marketing efforts. If not, how will you 
ever know whether or not it was wise? 

e Actively develop a network of trusted advisors. These 
could be local entrepreneurs, on-line forums, even 
bloggers you like. Everyone needs a Gerry or two. 
Advisors won't always agree with each other of 
course, but nothing beats running ideas past people 
you respect and who truly have your interests at 


heart, even if their advice ends up being wrong. 
Besides, no one knows what’s right, not even me. 


Seek advice that helps you become a better version of 


yourself, not advice that aims to change who you are. 


Being yourself is the only thing you’re going to be good 
at, anyway, which is what will give you the leverage you 


need to win. 
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Easy statistics for A/B testing and hamsters 


by Jason Cohen on April 6, 2009 


A/B testing tools often lie about whether something is 
“statistically significant.” Here’s an extremely simple, 
mathematically sound formula to compute it for yourself. 


So you’ve got your AdWords test all set up: Will people 
go for the headline “Code Review Tools” or “Tools for 
Code Review?” 


Gee they’re both so exciting! Who could choose! I know, I 


know, settle down. Welcome to A/B testing. 


Anyway, the next day you have this result: 


Variant A Variant B 
(“Code Review Tools”) (“Tools for Code Review”) 
Clicks: 31 19 


Is this conclusive? Has A won? Or should you let the 
test run longer? Or should you try completely different 


text? 


The answer matters. If you wait too long between tests, 
you're wasting time. If you don’t wait long enough for 
statistically conclusive results, you might think a variant is 
better and use that false assumption to create a new vari- 
ant, and so forth, all on a wild goose chase! That’s not 
just a waste of time, it also prevents you from doing the 
correct thing, which is to come up with a completely new 


test. 


Normally a formal statistical treatment would be too 
difficult, but I’m here to rescue you with a statistically 
sound yet incredibly simple formula that determines 


whether your A/B test results really are significant. 


Yl get to it in a minute, but I can’t help but include a 
more entertaining example than AdWords. Meet Hammy 


the Hamster, the probably-biased-but-incredibly-lovable 


tester of organic produce: 


Watch on YouTube 


In the movie, Hammy chooses the organic produce 8 
times and the conventional 4 times. This is an A/B test, 
just like with AdWords... but healthier. 


If you’re like me, you probably think “organic” is the 
clear-cut winner—after all Hammy chose it twice as often 
as conventional veggies. But, as so often happens with 
probability and statistics, you’d be wrong. 


That’s because human beings are notoriously bad at 
guessing these things from gut feel. Most people are 
more afraid of dying in a plane crash than a car crash, 
even though the latter is 1000x more likely. On the other 
hand, we’re amazed when CNN “calls the election” for a 
governor with a mere 1% of the state ballots reporting in. 


We also can’t distinguish between patterns and noise. 


Okay okay, we suck at math. So what’s the answer? 


Here’s the bit you’ve been waiting for!: 
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1 For the mathematical derivation, see the end of the article. 


Determining whether an A/B test is 
statistically significant 


1. Define N as “the number of trials.” 
For Hammy, VN =8+4=12 
[For AdWords, N = 31+ 19 = 50 


2. Define D as “half the difference between the ‘winner’ 


and the ’loser.”” 


—A 
For Hammy, D = -— =2 


31-19 | 
—< 


[For AdWords, D = 6 


3. The test result is statistically significant only if 
D? > N. 
(For Hammy, D? = 4, which is not bigger than 12, 
so it is not significant. 
[For AdWords, D? = 36, which is not bigger than 
50, so it is not significant. 


So your AdWords test isn’t statistically significant yet. But 
you let the test continue to run. The next day you find 31 
more clicks for variant A, and 19 more clicks for B. 


Rerunning the test, the measured difference is now 


significant: 
Variant Variant Stat 
A B N D D* Sig? 
Day one: 31 19 50 6 36 No 
Day two: 62 38 100 12 144 Yes 


A lot of times, though, you keep running the test and it’s 
still not significant. That’s when you realize you’re not 
learning anything new; the variants you picked are not 
meaningfully different for your readers. That means it’s 


time to come up with something new. 


<S 
SS 


SxS 
ressetonats 
SSS 

POSS 


When you start applying the formula to real-world exam- 
ples, you'll notice that when N is small, it is difficult— 
or even impossible—to be statistically significant. For 
example, say you’ve got one ad with 6 clicks and the oth- 
er with 1. That’s N = 7; D = 2.5; D? = 6.25. So the test 
is still inconclusive, even though A is beating B six-to- 
one. Trust the math here—with only a few data points, 


you really don’t know anything yet. 


The smaller the N, the bigger the difference needs to be, 
to be detectable. Specifically, results are significant only 
if the ratio between A and 8 is larger than a So for 
example, if N = 50, as it was in our “day one A/B test” 
example, the winning variant needs to be almost double 
the number of clicks as the losing variant in order to have 
a detectable difference, e.g. a conversation rate of 10% 
versus 5%. This is a huge difference; it’s great if you find 
something so dramatically better, but this is rare, and 
therefore you can almost never find a significant A/B test 
given only 50 clicks to analyze. 


When N = 100, the winner needs to be at least 50% 
higher than the loser (which it was by the second day in 
our example). It takes N = 1800 to detect the case 


where the winner is only 10% larger than the loser. 


And this is bad news for A/B tests, because often one 
variant isn’t better than the other by more than 10%, e.g. 
a “2.4% conversion rate” versus a “2.2% conversion 
rate.” What does this mean, especially if you don’t have 
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large N? It means you need to be seeking big differ- 
ences, not subtle ones. Test wildly different designs, 
rather than tweaks. Tweaks can only be tested when WN is 


enormous. 


I hope this formula will help you make the right choices 
when running A/B tests. It’s simple enough that you have 
no excuse not to apply it! Human intuition sucks when it 
comes to these things, and A/B testing tools often use 
misleading or incorrect math, so let this formula help you 


draw the right conclusions. 


Endnote for the mathematically inclined: 
The derivation 


The null-hypothesis is that the results of the A/B test are 
due to chance alone. The statistical test we need is 
Pearson’s chi-squared” 


2 Not the Student t-test as is commonly claimed by people online 
who have only a passing familiarity with statistics; the t-test is 
appropriate with continuous, normally-distributed random 
variables, whereas x” is appropriate for categorical random 
variables from independent trials and arbitrary probability dis- 
tributions, which is what an A/B test is. 


The definition of the x? statistic follows, where: 


m. = number of possible outcomes; 
Or 
Ex, = expected quantity of results in category k: 


2 w(O- Ky 


observed quantity of results in category k; 


In the simple case of a two-variant A/B test, m = 2. O; 
and QO» are the observed results, and definitionally 
N =O,+ Oy. The expected result under the null-hy- 
pothesis is that the quantities fall equally into each cate- 
gory, therefore Ey = Ey = N/2. 


Plugging this into the definition: 


The first numerator can be rewritten in terms of O, and 
Oz» by substituting N = O; + Ons, and this results in our 
variable D? as defined in the main text: 


N\? _ O1 — O2\* 
Ca a are 


We can repeat with the second numerator, and so the ex- 
pression simplifies: 


2-8? Or~ 9 
N N 


2 2 


D?2 
ae 
2 
2D”) 


Now that we have a simple formula for the chi-squared 
statistic, we refer to the chi-squared distribution to deter- 
mine statistical significance. Specifically: What is the 
probability this result would have happened by chance 
alone? 


Looking at the distribution at 1 degree of freedom, we 
must exceed 3.8 for 95% confidence and 6.6 for 99% 
confidence. For this simplified rule-of-thumb formula, I 
selected 4 as the critical threshold. Solving for D? com- 


pletes the derivation: 


x? >4 


(And if D? is more than double N, you’re well past the 


99% confidence level.) 
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Deriving the other statement in the article—that the ratio B- N _ J/N 
between the two variants needs to exceed a certain 
threshold to be significant—start with the boundary con- And so: 


dition of being significant, and derive the values of A 


Na 
and B in that case: A_ 2 VN 
B X_N 
2 
eee! N +2/N 


a= Nave VN -2 


Similarly, given that B = N — A: 
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Business Advice Plagued by Survivor Bias 


by Jason Cohen on August 17, 2009 


Advice from “successful entrepreneurs” might be unreliable 
due to Survivor Bias. What’s real, and what’s random? 


Google + tp 


Do you read articles written by a founder who failed 


three times, never finding success? 


No, because you want to learn from success, not hear 
about “lessons learned” from a someone who hasn’t yet 


learned those lessons themself. 


However, the fact that you are learning only from suc- 


cess is a deeper problem than you imagine. 


Some stories will expose the enormity of this fallacy. 


Bullet holes: A brain teaser 


During World War II the English sent daily bombing raids 
into Germany. Many planes never returned; those that 
did were often riddled with bullet holes from anti-air ma- 
chine guns and German fighters. 


Wanting to improve the odds of getting a crew home 
alive, English engineers studied the locations of the bul- 
let holes. Where the planes were hit most, they reasoned, 
is where they should attach heavy armor plating. Sure 
enough, a pattern emerged: Bullets clustered on the 
wings, tail, and rear gunner’s station. Few bullets were 


found in the main cockpit or fuel tanks. 
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The logical conclusion is that they should add armor plat- 


ing to the spots that get hit most often by bullets. But 


that’s wrong. 


Planes with bullets in the cockpit or fuel tanks didn’t 
make it home; the bullet holes in returning planes were 
“found” in places that were by definition relatively be- 
nign. The real data is in the planes that were shot down, 


not the ones that survived. 


This is a literal example of “survivor bias”’—drawing 
conclusions only from data that is available or convenient 


and thus systematically biasing your results. 


Doesn’t most business advice suffer from this fallacy? You 
read about successes but what about the businesses that 
“never made it home?” Like the downed planes, could 


failure contain more lessons than success? 


Burying the other evidence 


Scientific journals publish extraordinary results, so stud- 
ies whose results are statistically insignificance aren’t 
published. Rather, they are abandoned or silently stowed 


away in academic filing cabinets. 


For this reason, this practice is called the “file-drawer ef- 
fect,” and it’s a particularly insidious form of survivor 


bias because it is invisible. Peter Norvig sums it up nicely: 


When a published paper proclaims “statistically, this 
could only happen by chance one in twenty times,” it is 
quite possible that similar experiments have been per- 
formed twenty times, but have not been published. 


Pharmaceutical companies have exploited this effect to 
skew results intentionally. It’s gotten so bad that journals 


are calling for a public database to prevent fraud: 


More than two-thirds of studies of anti-depressants giv- 
en to depressed children, for instance, found the med- 
ications were no better than sugar pills, but companies 
published only the positive trials. 


If all the studies had been registered from the start, 
doctors would have learned that the positive data were 
only a fraction of the total. 

—Washington Post 


Doesn’t most business advice suffer from this fallacy? 
Harvard Business School’s famous case studies include 


only success stories. To paraphrase Norvig, what if twen- 
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ty other coffee shops had the same ideas, same prod- 
uct, and same dedication as Starbucks, but failed? 
How does that affect what we can learn from Starbucks’s 


success? 


Experimental proof of ESP 
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Dr. Joseph Rhine brought the rigor of experimental psy- 
chology to the study of the paranormal, and ESP (Extra 
Sensory Perception) in particular. He made waves in the 
1930s with controlled experiments testing whether a per- 
son was able to predict the order of the cards in a shuf- 
fled Zener deck (with symbols like circle, square, star, 


and wavy lines). 


In a typical experiment, 500 people are screened for 
“strong telepathic ability,’ measured by significantly 
above-average performance in a 25-card deck. Those se- 
lected are tested again; most drop away. Tested a third 
time, perhaps one person passes again and we conclude 
that such a repeat performance is statistical evidence of 


genuine ESP 


To see why this is just a different face of survivor bias, 
consider the following experiment. I believe some people 
are “heady” when it comes to coin-flipping—getting 
heads more often than chance alone would suggest. So I 
put 1000 people in a room and tell them to flip a coin ten 


times. Sure enough, a woman named Margaret makes 


“heads” ten times in a row! The chance of her getting 
heads ten times in a row is only 1-in-1024, so I conclude 


Margret has special abilities. 


Actually that last statement is true but misleading. The 
chance that Margaret would flip ten heads in a row is 1- 
in-1024, but that wasn’t the experiment I ran, was it? I 


let 1000 people flip and “found” Margaret in the crowd. 


The chance that somebody in a crowd of a thousand 
would flip heads ten times is a whopping 62%! Because 
so many people are attempting the feat, some normally- 
unlikely events will happen. This isn’t a test of Margaret’s 


abilities at all! 


Doesn’t most business advice suffer from this fallacy? Take 
me for instance. I’ve started three! consecutive successful 
companies, so that’s proof that I know what I’m doing 
and that you should do everything I say, right? Except 
maybe I’m just the one in the crowd who guessed right 
on the Zener cards three times, and there’s no reason to 


believe I would be successful a fourth time” 


1 author’s note in 2023: Now four with WP Engine; with 
200,000 customers and hundreds of millions in ARR, and prof- 
itable, it’s an uber-Margaret! 


2 Author’s Note in 2023: Looking back fourteen years, having 
done it again, I still wouldn’t change a word. Maybe I’m still 
Margaret. Perhaps I’ve earned additional benefit of the doubt, 
but the survivor bias warning remains. 
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Specific examples of survivor bias in 


§ 


me 


business advice 


So far I’ve been asking rhetorically whether survivor bias 
might be severely skewing business advice. Steven Levitt 
(of Freakonomics fame) investigated this question 


directly. 


He was reading Good to Great by Jim Collins, a book that 
analyzed eleven companies that were mediocre—just 
pooping along—but then transformed themselves into 
stock market sensations. A conclusion was that the com- 
mon trait was a “culture of discipline.” This book has sold 
many millions of copies, so it’s a good example of popu- 


lar writing on business advice. 


One of the eleven “great” companies was Fannie Mae, 
and Steven Levitt was reading this book just as Fannie 
was collapsing in financial disaster. Hmm, he thought, I 


wonder how those other “great” companies are doing. 


Turns out, had you invested in those eleven companies in 
2001 (when the book came out), your portfolio would 
have underperformed the S&P 500! (Fannie Mae wasn’t 
even the only case of total disaster—also extolled was the 


now-bankrupt Circuit City.) 


Why didn’t these companies continue to succeed? It turns 
out Jim started by combing through 1435 companies 
looking for good candidates for the book, and picked 


eleven. It’s the ESP experiment all over again! 


On top of that, Jim doesn’t bother asking whether any of 
the 1424 other companies also displayed a “culture of 


” 


discipline.” Maybe that’s something that many public 


companies have regardless of performance. 


Is this book an aberration? Nope, Steven investigated an- 
other business book from the 1980s—In Search _of 
Excellence—and found the same effect. 


Steven then comes to the same conclusion that I’m com- 


ing to: 


These business books are mostly backward-looking: 
what have companies done that has made them suc- 
cessful? The future is always hard to predict, and un- 
derstanding the past is valuable; on the other hand, the 
implicit message of these business books is that the 
principles that these companies use not only have 
made them good in the past, but position them for con- 
tinued success. 


To the extent that this doesn’t actually turn out to be 
true, it calls into question the basic premise of these 


books?, doesn’t it? 


31 personally love Good to Great anyway, because I think the 
ideas in there are timeless and excellent. 


Oops, did I just invalidate this entire 
website? 


Lately I’ve been wondering if a lot of business advice— 
both mine and others—is really a case of survivor bias. I 
mean, I didn’t start out at Smart Bear with a load of phi- 
losophy and a fixed idea of who the customer was or 


even what the products would be. 


How do I know this post-hoc philosophy and advice isn’t 
just a case of survivor bias? Am I not like the ESP-savant, 
successful not by force of nature but by simple chance of 


surviving? 


Or perhaps I’m like Dr. Rhine the ESP experimenter— 
convinced I’ve discovered something important with “ob- 


jective measures of success”—and yet I’m actually living 
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in an egotistical, possibly even narcissistic dream world. 


More to the point, how can you, dear reader, ascertain 
whether my articles or any advice from anywhere suf- 


fers from this fallacy? 


In the end of course you don’t know. But here’s some- 
thing: Just the fact that you’re aware of survivor bias 
means your're less likely to be taken in by it. So, reading 
this article has helped a little. 


Beyond that, prefer advice that makes you think rather 
than giving you answers, forces you to answer tough 
questions, and causes you to extend your existing 
strengths, and become a better version of the person you 


already are. 


Printed from: A Smart Bear 
https://longform.asmartbear.com/survivor-bias/ 
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Rich vs. King in the Real World: Why I sold my company 


by Jason Cohen on October 19, 2009 


Reflecting on selling Smart Bear in 2007, offering insights for 
entrepreneurs facing similar decisions. 


I sold my company, Smart Bear, in December of 2007. I 


haven’t talked about it at all on this blog, and it’s time! 


spill my guts about the whole affair. 
1 This article was written in 2009. 


You'd think selling a company would be a glamorous, ex- 
uberant experience, but I was surprised at the reactions I 


got. These are actual quotes: 


e “How could you sell your baby? I’m shocked.” 
e “Tl thought you said things were going well. Hmm.” 
e “You’re such a sell-out! You used to be one of the 


few cool people I knew.” 


Interestingly, 100% of the negative reactions were from 
people who had never started their own company. But 


that doesn’t make them wrong, and it doesn’t make their 


words sting less, especially when they’re your friends. 


Now that almost two years have passed, I can relate ex- 
actly why “selling my baby” was right for me. 


Hopefully this thought process is interesting to you and 
possibly useful in the happy event that you’re faced with 
the same choice, but the truth is I just need to get this off 


my chest. 


I need to explain to those who still consider me a sell- 


out. 


You may have heard Noam Wasserman’s “Rich or King” 
choice: Company founders are either in it for the money 
(“Rich”) or in it to build a lifestyle and personal identity 
(“King”). FogCreek and 37signals are built to be “King;” 


all venture-funded companies are built to be “Rich.” 


Noam says that successful founders make the “Rich or 
King” decision up front, and that though it doesn’t matter 
which path you take, you must be consistent in your ac- 
tions. You can’t mix “be king” tactics with “get rich” end 


goals. 


Except I did mix “Rich” and “King,” and it worked. 
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See, it’s good to be “King,” but what do you do when 
you're at Trudy’s “North Star” Tex-Mex Restaurant tuck- 
ing into a Chile Relleno (with salsa verde, black beans, 
and the ground beef filling), and the guy across the table 
looks you in the eye and offers you enough money that 


you never have to work again? 


I was always in it for the money, especially in the form of 
acquisition. Everyone who came to work at Smart Bear 
was indoctrinated with this attitude in no uncertain 
terms; even before hiring someone, I would tell them 
that we’re here to make money, and if someone offers to 
buy the company someday, I’m going to sell it, and all of 


us will make money. 


Profit was the rule behind every choice we made. 
Although the end goal was always acquisition, my atti- 
tude was (and still is) that the best way to get yourself 
acquired is to be profitable Profits prove the business is 
operating well. Profits validate the market. Profits make 
minimum valuation easy. Profits mean the buyer converts 
balance-sheet money into bottom-line profit-and-loss 


money—a trade every large company wants to make. 


2 Editor’s Note: At this time the blog was written exclusively for 
bootstrappers with small companies. Obviously there are many 
examples of companies sold on the basis of growth or “daily ac- 
tive users,” but small self-funded companies are most attractive 
when they are cash machines. 


Most of all, profits mean you don’t need to sell, which 
gives you the ability to walk away from a deal. You have 
little negotiating power in any deal unless you can happi- 


ly walk away. 


On the other hand, I knew I would only be happy build- 
ing a genuine, great company, where the product solves a 
real pain, where customers are given white-glove service, 
where “tech support” is the only sales force, where we 
leave the world a little better than we found it, and 
where every employee is smart and gets things done and 


is trusted with any decision. 


And I wanted the ego-inflating trappings of running a 
company. It’s cool at parties to say “I run my own com- 
pany.” I wrote a book that got so popular (in my little 
corner of the world) that people would bring it up to me 
to sign. (We gave the books away for free so the joke was 
that by signing I doubled its value.) When I walked onto 
a tradeshow floor it was like Norm on Cheers—I knew 
everyone and they knew me. I got to present at cool 


venues like the Business of Software Conference. 


And I write this blog, shamelessly exploiting the fact that 
Smart Bear (and two other companies) were successful 


to convince you that I’m worth reading. 


In short, although the goal was “Rich,” I achieved it by 
behaving like the goal was “King.” I don’t know why 
people find this contradictory; after all, acting like “King” 
means building a long-term, sustainable business, and 


that’s exactly the kind of business that gets acquired. 


Still, because “King” was enjoyable and Smart Bear was 
profitable, I still need to explain why becoming a “sell- 


out” was the right choice. 


The first thing to understand is the non-linear relation- 
ship between “cash in personal savings” and “financial 


freedom”: 


How cash in the bank affects your lifestyle: It's not linear. 


Private jet 
Never have to 
work again 
Never look at 
the right side 
of a menu 
Own your 
house 
Cantlose 
your job 

$1k $100k $? mil $50mil 

(Freedom) 


Cash in the Bank 
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There’s a line you cross where your savings alone will 
fund a reasonably lavish lifestyle. At the risk of sounding 
like George Bush, this is a Freedom Line—freedom from 
restrictions about what you can do with your life, family, 


and career. 
My observations: 


1. A movement from left of the line to right of the line 
changes your life fundamentally, giving you the 
freedom to do whatever makes you happy, forever. 

2. If youw’re crossing from left to right, it doesn’t matter 
how far to the right you go. (Sure, $100m is a 
different lifestyle than $10m, but it’s not as critical to 


lifestyle or happiness as just crossing the line.) 


(1) is what was offered to me at Trudy’s Tex-Mex. (2) 
means it almost didn’t matter what the offer was, so long 


as it was big enough. 


Some people gave me a hard time about (2). The typical 


argument was: 


Your company is growing 100% year over year. It’s 
profitable and throwing off cash. Why not wait another 
year and let revenues double again, which will make 
the company six times more valuable (assuming 3x 
revenue valuation, a reasonable ballpark for a growing 
software company” ). 


3 Editor’s Note: This was written in 2009; In 2023, a 5x-10x mul- 
tiple on this type of company at that growth rate and profit is 
reasonable. 


Here’s the best analogy I’ve come up with to describe 
why this is flawed logic. It’s called the Box Game: 


Imagine I have two opaque boxes. Box A contains $10. 
Box B has a 50% chance of containing $20, and a 50% 
chance of containing nothing at all. You pick either box 


and take whatever’s inside. Which box do you pick? 


Of course statistically there’s no difference, so this isn’t a 
question of math or economics or intelligence; it’s a mea- 


sure of your attitude towards risk. 


Most people pick box B. After all, the difference between 
$10 and $20 is trivial and it’s more fun and exciting to 
pick B. 


But what if the numbers were different? 


Now box A holds $10,000,000. Box B either holds 
$20,000,000 or nothing, 50/50 chance. Which do you 
pick? 


You pick box A. Of course! Because it moves you from the 
left of the line to the right. And because a “chance of 
moving even further” isn’t worth giving up the certainy of 
that life-altering event. 


This is my argument in favor of #2 and against “wait and 
see.” This is why I sold. 


In my case, the correctness of my choice was made 
painfully clear by the economic crash in 2008. Had I held 
out for “another year and far more money’—box B—I 


would have found an empty box. 


I know this for a fact—another company (can’t say who, 
sorry!) was offered a deal at the same time I was. This 
founder wanted to roll the dice (box B) and delayed the 
buyer. Two quarters passed and revenue failed to grow; 
the buyer nixed the deal. Months later with the recession 
in sight, the founder approached the buyer again, this 


time willing to accept a low offer. The buyer refused; that 
ship had sailed. 


There are those for whom this calculus doesn’t apply be- 
cause they want to be “King” no matter what. I'll bet 
Jason Fried wouldn't sell 37signals for $100,000,000; 
neither would Joel Spolsky sell FogCreek. Are Joel and 


Jason being irrational? Of course not. But neither was I. 
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; As of December 2007, I have the freedom to work on any 
a Codie Sanchez @ 


ey aa leearienie? project I want for the rest of my life while simultaneously 
Evel prounaeriigats providing for my family, never again worrying about bills, 
"| love my company. These people are my family and | will run this debt, having a place to sleep, or sending our daughter to 


business forever." 
any college she wants. 


Every Founder in Year 8: 

I can stay home with my wife and new baby girl for as 
"| hate my life and my employees hate me. Please buy my business and . . . . 
let me go away for a very long time." long as I want, having all the precious time and experi- 


ecieeexe ences and memories that they say money can’t buy. 


1:27 PM - Dec 26, 2023 - 59K Views A a ‘ 
But, in the sense of securing that freedom, it can. 


b 
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And by crossing the line, I did. 
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Impostor Syndrome: Why I felt like a fraud, and how I overcame it 


by Jason Cohen on January 18, 2010 


Most high-performing people experience Impostor 
Syndrome. | did too. When you understand the cause, you 
can defeat it. 


“T feel like a fraud. I’ve been at this for 16 years and I 
still feel like a fraud. I’m just waiting for the day they 
see through the facade, but they keep coming back 
every year.” 

—Jason Young 


Ah yes, the awe-inspiring words of confidence from the 
seasoned entrepreneur. My friend Jason intended this as 
soothing words of solace during (one of my) periods of 


personal freak-out while Smart Bear was in its infancy. 


I felt like a fraud every day. Here I was, selling a wobbly, 
buggy tool and pawning myself off as an expert in a field 
that didn’t exist! Every second I felt like I was putting 


one over on the world. 


1 Our software was the first commercial tool for peer code re- 
view. Now a standard practice for many developers, and a stan- 
dard feature in many development tools, at the time we were 
creating a market that didn’t exist. 


I would explain how our tool “cuts code review time in 
half,” but was that actually true or had I just repeated the 
argument so many times that I stopped questioning it? I 
would orchestrate purchases, but should I be handling 


large sums of money with no knowledge of accounting, 


cash-flow, invoicing, purchase orders, all while being re- 
lentlessly belittled by Accounts Payable? I would instruct 


customers on “best practices” for code review, even pub- 
lishing a whole book about it, but who am I to tell other 


people how to critique code? 
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Aren't I too young? Isn’t the tool too crappy to charge 
for? Aren’t I too inexperienced? Don’t I need an MBA or 


at least some sales training? 


Is Smart Bear a “real company?” What does that even 


mean? 


Objectively, and with hindsight, my feelings were mis- 
placed. The tool really did save time and headache; cus- 
tomers said so. As much as I doubted the title “Code 
Review Expert,” I had developed more experience with 
more teams in more situations than any one person could 
(because everyone else was busy doing their actual jobs). 
And sales isn’t as mystical and unknowable as I feared. 
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Still, emotions don’t respond to logic. Jason was telling 
me that these feelings don’t go away, even when they 


ought. 


The other thing he was saying is: You’re not alone. It’s 
not just business founders. Mike Meyers said “I still be- 
lieve that at any time the No-Talent Police will come and 
arrest me.” Jodie Foster said “I thought it [winning the 
Oscar] was a fluke. The same way as when I walked on 
the campus at Yale. I thought everybody would find out, 
and they’d take the Oscar back.” June Huh, who dropped 
out of high-school to become a poet, and later won the 
Fields Medal—mathematics’ highest honor—says “Of 
course you are happy, but deep down, you’re a little bit 
worried that they might eventually figure out that you’re 
not actually that good. I am a reasonably good math- 
ematician, but am I Fields Medal-worthy?” 


“I think instead of worrying about why people don’t believe in 
you, we should worry about why you don’t believe in yourself.” 


The name for this is Impostor Syndrome. Studies show 
that “40% of successful people consider themselves 
frauds” and that “A staggering 84% of the entrepreneurs 
and small business owners surveyed had scores indicat- 
ing they feel like an imposter at moderate, frequent, or 
intense levels.” Ask any small business coach; they'll con- 
firm how prevalent these feelings are. It’s also common 
with PhD candidates. People constantly complain of it on 
Twitter, and that’s just the <1% who are (a) on Twitter 


and (b) willing to admit it. 


Although not an official psychological disorder, and gen- 
erally not crippling, if you have these feelings it’s useful 


to know that it’s common and there’s something you can 


do about it. 
See if these sound familiar: 


e You dismiss compliments, awards, and positive 
reinforcement as “no big deal.” 

e You are crushed by mild, constructive criticism. 

e You believe you’re not as smart/talented/capable as 
other people think you are. 

e You worry others will discover you’re not as 
smart/talented/capable as they think you are. 

e You think other people with similar jobs are more 
“adult” than you are, and they “have their shit 
together” while you flounder around. 

e You feel your successes are due more to luck than 
ability; with your failures it’s the other way around. 

e You find it difficult to take credit for your 
accomplishments. 

¢ You feel that you’re the living embodiment of “fake it 


until you make it.” 


But wait, how can this be? This overwhelming lack of 
self-confidence is the opposite of the traditional en- 
trepreneurial stereotype. Don’t founders forge ahead 
even when others say success is impossible? Doesn’t a 
founder invent a new product based on her confidence 
that others will want it? Doesn’t the very idea of starting 
your own company scream “I’m doing it my way, and my 


way is better?” 


Here’s why it makes sense. Consider what it means to be 
a perfectionist. The perfectionist sees flaws in everyone 
else’s work; there’s always a way to make it better—her 
way. She doesn’t respond well to authority dictating how 
things must be; neither is she comfortable delegating to 
those who (by her definition) clearly don’t care as much 


as she does. 


Sounds like the stereotypical attitude of the arrogant 
startup founder, but wait! At the same time, the perfec- 
tionist is never happy with her own work either, seeing 
(inventing?) a never-ending stream of flaws that require 
attention. No matter how highly others regard her work, 


the perfectionist insists it’s incomplete and unsatisfactory. 
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She can’t accept the idea that others would be impressed 
with her accomplishments, because to her eyes they’re 
mediocre works-in-progress. She worries that one day 


they'll realize she’s right. 


Our entrepreneurial motivation is not confidence, it’s an 
insatiable desire to do it our way. Doing it our own way is 
not the same as being confident that it is the correct way. 
It’s about never accepting any idea as being best, even if 


you like your current ideas better than everyone else’s. 


Can these feelings be constructive? Yes, if they're a sign 
that you’re striving to learn and improve. As Andy 
Wibbels says: 


If I don’t feel like a fraud at least once a day then I’m 
not reaching far enough. 


If you aren’t scared shitless then why bother? 


Here’s what it looks like when you’re channeling these 


self-doubts into something constructive: 


¢ I doubt my title as “expert,” so every day I read, 
write, and immerse myself in my field. 

¢ I doubt the quality of my software, so I fix bugs as 
fast as possible, I write unit tests proactively, and I 
thank my customers for their patience. 

e I doubt I deserve my reputation, so I work hard to 
earn it. 

e I’m not as good as I want to be at speaking/writing/ 
programming/designing/managing, but I can see 
myself slowly improving. 

¢ I’m not a “real company” yet, so I concentrate on 
making my customers successful, so they don’t care 


about corporate size or structure. 


On the other hand, here’s what it looks like when these 


doubts are harming you: 


e I doubt my title as “expert,” so every night I worry 
about what will happen when I’m discovered as a 


fraud. I’m absent-mindedly looking for trivially-easy 


jobs I could take where this pressure won't exist. 
(Looking for an “escape-hatch” is a well-documented 
behavior.) 

e I doubt the quality of my software, so I spend lots of 
time covering it up with graphic design and heavy 
sales pitches. 

¢ I doubt I deserve my reputation, so I live in constant 
fear of exposure. I can’t sleep at night and I loathe 
myself for lying. 

¢ I’m not as good as I want to be at speaking/writing/ 
programming/designing/managing, so I go out of my 
way to avoid any of it, and feel like a trapped animal 
when I’m forced to do it. 

e I’m not a “real company’ yet, so I feel guilty every 
time someone gives me money or believes anything I 


say. 


If you’re letting these feelings get to you too, at least rec- 


ognize it so you can deal with it logically. 


“Turns out my imposter syndrome was pretty much 
right on.” 


Here’s some specific ways to answer the Impostor 


voice in your head: 


.-yet 

Whatever it is you think you’re not good enough for, 
append the word “yet.” I can’t do this job... yet. This 
article isn’t good enough... yet. I don’t deserve the ti- 


tle I have... yet. 
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“Am I a curious student?2” 

A: When you meet a student, who is honesty, energeti- 
cally, and humbly pursuing learning and growth, do 
you think “That’s awesome, good for them, how can I 
help,” or do you think “They’re an impostor for not al- 
ready being an expert?” So, when you act as a curious 
student, that’s not being an impostor, that’s the state 
of being a student. Which is awesome. 


2 Inspired by Mihir Patel’s article 


“Do I deserve to be peers with my peers?” 

A: Yes, unless you think Neil Gaiman doesn’t deserve 
to be at the same party as Neil Armstrong or vice ver- 
sa, because they feel the same way, and you know 


they’re both wrong. 


“Do I deserve the [title/position/responsibility] I 
have?” 
A: Do you think some idiot gave you a hand-out for 


nothing? If not, then yes, you deserve it. 


“Sometimes I feel stretched and that I don’t know 
what I’m doing.” 

A: That is the human condition, for humans who 
strive. Only someone who is ignorant and foolhardy 
doesn’t realize there’s more to know, more to learn, 
and to be careful with confidence. If Mike Meyers isn’t 


sure whether he’s funny, then it’s just the human con- 


dition. Even if sometimes you are stretched too far, 
that’s the normal condition for striving. It’s something 
to embrace and work on, not evidence that you 
shouldn’t be doing it at all. Unless you think “learning 
and growing” is something you should never do, or if 
you think “learning and growing” should never feel 


uncomfortable. 


And when logic fails, maybe this will help: 

You believe that Mike Meyers and Jodie Foster are talent- 
ed, right? And June huh deserves that medal? You might 
even believe that I’m an expert in peer code review and 
startups in general. Yet we doubt ourselves every day. 


And we're wrong. 
So if we’re wrong, you're wrong too. 


Don’t stop striving, just stop holding yourself to an im- 


possible standard, and don’t worry what other people 
think. 


They’re not thinking about you at all. 


Many thanks to Matt Cohen and Scott W. Bradley for contributing 
examples. 
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by Jason Cohen on March 1, 2010 


What if your company could have only one single advantage 
over the competition? This exercise will make your 
positioning and strategy stronger. 


“I’m sorry; pick up a card from the pile in front of us 
and hope that it will be something you need.” 


If your company were allowed only one advantage over 
the competition, what would a sales call look like, start- 
ing with your 30-second pitch, then dealing with skepti- 
cal questions, earning this potential customer’s interest, 
respect, and eventually money, all with only one 


advantage? 
Impossible? Pointless? Neither! 


You should go through this exercise because this skill 
is valuable in every sales call. Sometimes you're de- 
fending the few advantages you have over a specific com- 
petitor. Sometimes you're arguing the virtues of small 
businesses over large ones. Sometimes you’re defending 
your product against what the potential customer per- 


ceives as a glaring lack of functionality. 


Hanging your hat on just one advantage that you can own 
completely is stronger than diluting your message by 


making many assertions. 


It’s not just in face-to-face sales calls either. Your home- 
page becomes laser-focused. Your advertisements become 
pointed, powerful, pithy, and other words starting with 
“p.” Your 30-second pitch becomes compelling. You know 
what to blog and Twitter about. Your 5-minute product 
demo reinforces a singular point. Everyone knows who 
you are and where you stand. Especially Carol: Your ex- 


tra-special ICP (Ideal Customer Profile). 


At least on this one point, you’re untouchable. Doesn’t 


that sound nice? It is nice. 


But don’t you need lots of advantages to overcome 
sales objections and competition? No. Let’s see how to 
riff off single advantages, using them to answer a range 


of skeptical questions and concerns. 


If nothing else, this should get your juices flowing and 
make future sales calls and marketing messages more 


effective. 


Most Expensive 


You: We’re the Cadillac tool—the most expensive, but 
also the best. I know, “most expensive” doesn’t automati- 
cally imply “best!” But in our case you get what you pay 
for. 


Customer: Hmm, I don’t know, budgets are tight. We’re 


thinking of going open-source—it’s free. 


You: Open-source is free like puppies are free. You 
don’t write a check to get it, but you have to support it 
for life. Your employee’s time is not free. Working around 
bugs is not free. Having nothing but the Web-ef-ties 
Internet to rely on for tech support is not free. See, we 
don’t line our pockets with that revenue, we spend it 
making you maximally effective. We answer the phone 
on the first ring. When you have a problem, we connect 


you directly with developers instead of hiding behind off- 
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shore Level 1 support. We’ll stay on the line with you at 
3am as you work through a problem. We'll do a confer- 
ence call helping you through best-practices on using the 
tool for your specific purpose. We do things an open- 


source project cannot do. 


Customer: OK, that’s useful. But BigCorp offers 24/7 tech 


support too and they have consultants. 


You: It’s quality, not quantity. Let’s get specific. We em- 
ploy actual software developers for Level 1 tech support 
and email, so you’re talking to someone who not only 
can answer every question but can even read the code to 
get answers. You're talking to someone who has the pow- 
er and ability to change the code to fix a bug or add a fea- 
ture. That’s an inside track that no big company will of- 
fer. And consultants? Our consultants write blog posts 
about best practices. Our consultants live with the entire 
team every day. Our consultants train with the top ex- 
perts in the field, who we can afford because we’re the 
Cadillac. You’re not getting someone who realized they 
can turn a buck installing high-priced software—you’re 
getting true experts giving you insight that only we can 
provide. 


Customer: I’m also checking out SimpleCo’s tool. They’re 
much cheaper, and although they don’t have as many fea- 


tures, it seems simpler to use. 


You: Just because they do less doesn’t mean they’re easier 
to use. For example, one of the reasons we’re expensive is 
that we integrate with 20 other software packages. That’s 
great for you, because it means we interoperate with 
more of your other tools—including that tool you’re go- 
ing to buy next year but you don’t know it yet. But it’s 
not more complex for you, because if you don’t use an in- 
tegration, that has zero impact on your day-to-day use. 
There’s a myth that “more features are always more 
complex,” but that’s just bad user interface design. 
And yes, you guessed it, we can afford awesome user in- 
terface experts who help us avoid those mistakes. 


Customer: But still, even if I agree with all that, I still 


have to justify the budget today. 


You: When you factor in the cost of the tool, also factor in 
the cost of failing to be successful with the tool. To spend 
many months installing, integrating, training, learning, 
customizing, fighting, on the line with tech support, only 
to have it fail in the end—having to rip it out, then go 
through the whole process again with a new tool. 
Multiply that by the chance the tool will indeed fail. Sure 
it’s possible that any tool could fail, but with us—more 
features, better support, expert help—it’s less likely to 
happen. Oh, and besides the catastrophic expense, 
what’s the effect on your personal career? What’s best 


for you and your company is to bet on the best. 


(Will this work? I don’t know, but at least the talk-track is 


clear.) 


Ay eRea 


“Thousands of times he’s asked people to ‘pick a card, 
any card,’ and not once have I been picked.” 


Obsessed with Quality 


You: Software is so crappy nowadays, we expect failure. 
We expect bugs. We expect to be helpless, to just have to 
“deal with it.” At AwesomeCorp, we say that’s 


unacceptable. 


Customer: Yeah.... so you’re saying you have no bugs at 


all? 


You: No, I’m saying we’re maniacal about finding bugs, 
and when you find one we're incredibly fast at fixing 
them. It’s not unusual to have a fix in under 24 hours. All 
software can have bugs, but no one is more committed to 


fixing them. 
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Customer: Well if that’s true, that’s good. But I’m current- 
ly trialing BigCorp’s tool and they have more features 
than you. I don’t know which we'll need, but I have a 


problem buying a tool that doesn’t do much. 


You: It sometimes sounds like “more bullet points of 
more features” is always better, but you and I have used 
software that claimed to have lots of useful features that 
didn’t really work in reality. Usually the more features a 
product has, the worse each feature is. Try uploading a 
huge file to BigCorp’s tool—oops, it breaks! To us, saying 
you have a feature when in reality it’s full of holes is dis- 
honest. We’d rather know we have fewer features that we 
actually stand behind rather than claim to have features 


that are just incomplete. 


Customer: OK, I can appreciate that, but what about 
OpenSourceOrg’s tool? I know it doesn’t do quite as 


much as yours, but free is free! 


You: Yup, free is free... until you run into a bug. It’s free 
until it has a security breach that you didn’t find out 
about for two years. It’s free until you notice there’s in- 
correct data floating around. It’s free until you need 
something and there’s no one to ask. Of course they say 
“you can fix that bug yourself” or “you can add that fea- 
ture yourself,” but that ain’t free! And even if you invest 
the effort, if they don’t accept your patch you'll constant- 
ly have to re-patch when you get updates. Bugs are a re- 
ality, and that’s when open-source starts to become non- 
free in a hurry. We, on the other hand, never charge you 
for bug fixes, even years later, because we’re 100% com- 


mitted to quality code. 


(Will this work? I don’t know, but at least the talk-track is 


clear.) 


Small Company 


You: If you haven’t worked with a small company before, 
you're in for a nice surprise: Smart people you can actu- 
ally talk to, people who care about what you need, peo- 
ple willing to go out of their way to make sure you’re 
successful. 


Customer: I get that, but little companies fail all the time. 
How do I know you'll be around to give me that great 


support? 


You: You say that as if big companies are stable during re- 
cessions and accounting scandals! You say that as if big 
companies don’t cut entire product lines if they're not 
profitable, or sell them off even if they are. It’s impossible 
to know when a big company is about to discontinue 


your product, and it happens all the time. 


Customer: You say you have great support, but BigCorp is 
the one with 1000 developers and support engineers will- 


ing to help me. 


You: “Willing” to help you perhaps, but able? Typically 
the software developers are shielded by “Level 1 
support”—people without power, certainly not the power 
to get your feature requests into the pool. In fact, 
wouldn’t you agree most tech support feels like a shield 
rather than a help? Even if you get a bug into the pile or 
a feature onto the list, big companies release new ver- 
sions infrequently, so you might have to live without it 
for a year. Not us. You get to complain directly to the en- 
gineers who can fix the problem in weeks—sometimes 


days. 
Customer: But they have 24/7 support. Do you have that? 


You: No, we don’t pay folks we’ve never met 
$1.25/hour to answer the phone at 3am PST so they 
can tell you to reboot your machine and RTFM. 
Instead we pay actual software developers $75/hour to 
talk with you in person about exactly what’s wrong, ei- 
ther solving the problem or getting it fixed ASAP 
Sometimes we even write special code just to get you 
running again, tiding you over until a proper fix is re- 


leased. Try getting that from BigCorp! 


Customer: Well if that’s true, that’s good. But BigCorp 
also has more features than you. 
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You: Do you really want your tools to have “more fea- 
tures,” or is it really that you want your specific needs 
met, and “more features” could potentially mean that 
more of your needs are met? We believe the point of soft- 
ware is to solve your problems and make your life better 
without incurring too much new expense in time and 
money. Even assuming BigCorp’s has one or two features 
you like today, what about in six months when you’re 
deep into the tool and realize there’s 10 more things 
you really want? Do you expect them to add half of 
those to their next release? Because that’s exactly what 
we're going to do—hold proactive meetings to find out 
what you need to be most productive, and agree to add 
those as soon as we can. Don’t ask “Which tool will satis- 
fy my needs today,” ask “One year from now, which tool 
will be satisfying my needs, including the ones I can’t 


foresee?” 


(Will this work? I don’t know, but at least the talk-track is 


clear.) 


Why try to defend ten points when you only need one 


or two to make your case? 


Why not focus your message, focus your behavior, focus 
your UX, focus your product, focus your target customer, 


and focus your sales pitch? 


It’s already hard enough to stake out a niche in this mas- 


sive world! Don’t dilute your message. 
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The Pattern-Seeking Fallacy 


by Jason Cohen on June 14, 2010 


We humans are terrible at discerning patterns from 
randomness, and in marketing data we unwittingly find 
“insights” that are actually noise. Here’s how to fix that. 


Which of these sets of dots were placed randomly? 


We think “right,” but the answer is “left.” Humans are bad at 
seeing the difference between patterns and randomness. 


(From Steven Pinker’s The Better Angels of Our Nature 2011) 


What do these have in common? 


e “This pitcher has retired 5 of the last 7 batters.” 

e “We tried 10 AdWords variants and combination D is 
the clear winner.” 

e “The Bible Code predicted the Sept 11 attacks 5,000 
years ago.” 

e “We sliced our Google Analytics data every which 
way, and these 4 patterns emerged.” 


All are examples of a common fallacy that I’m dubbing 


the “Pattern-Seeker.” 


You probably laugh at Nostradamus, yet it’s likely you’re 


committing the same error with your own data. 


Patterns in Chaos 


It’s commonly said that basketball players are “streaky”— 
they get on a roll hitting 3-pointers (have a “hot hand”) 
or develop a funk where they can’t seem to land a shot 
(“gone cold”). These observations are made by fans, an- 


nouncers, pundits, and the players themselves. 


“He can play like this, but he'll only 
be able to give 105%.” 


In 1985 Thomas Gilovich tested whether players really 


did exhibit streaky behavior. It’s simple—just record hits 
and misses in strings like: HMHHMMMMHMMHH, then use 
standard statistical tests to measure whether those 
strings are typical of a random process, or whether there 


is something more systematic going on. 


Turns out players are not streaky; simply flipping a coin 
produces the same sort of runs of H’s and m’s. The scien- 
tists gleefully explained this result to basketball pundits; 
the pundits remained unimpressed and unconvinced. 


(Surprised?) 


So they tried the same experiment backward: They creat- 
ed their own strings of H’s and m’s with varying degrees 
of true streakiness and showed those to pundits and fans, 
asking them to classify which were streaky and which 
were random. Again they failed completely, much like the 
“star field” test at the top of the article. 


We perceive patterns in randomness, and it extends be- 
yond casual situations like basketball punditry, plaguing 
us even when we're trying to be intentionally, objectively 


analytical. 
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Take the “interesting statistic” given by the baseball an- 
nouncers in the first example above. Sure the last 5 of 7 
batters were retired, but the act of picking the number 
7 implies that number 8 got on base, and so did the 
one before the sample window. So it’s at least “5 of 9,” 
not “5 of 7,” but that doesn’t sound as impressive, even 


though it’s the same data. 


Unlike the basketball example, the baseball announcer’s 
error runs deeper, and following that thread will bring us 


to our own marketing data and the heart of the fallacy. 


Seeking combinations 


Baseball statisticians record a dizzying smorgasbord of 
statistics that announcers eagerly regurgitate. Maybe it’s 
because baseballers are a little OCD (evidenced by pre- 
bat and pre-pitch rituals) or maybe it’s because they need 
something to say to soak up the time between pitches, 


but in any case the result is a mountain of data. 


Announcers exploit that data for the most esoteric of 


observations: 


“You know, Rodriguez is 7 for 8 against left-handed 
pitchers in asymmetric ballparks when the tide is going 
out during El Nifio.” 


This is the epitome of Pattern-Seeking—combing 


through a mountain of data until you find a pattern. 


Some statistician examined millions of combinations of 
player data and external factors until he happened across 
a combination which included a “7 of the last 8,” which 
sure sounds impressive and relevant. Then he proudly 


delivered the result as if it were insight. 


So what’s wrong with stumbling across curious observa- 


tions? Isn’t that how you make unexpected discoveries? 


No, it’s how to convince yourself you’ve made a discovery 
when in fact you’re looking at pure randomness. Let’s see 


why. 


Even a fair coin appears unfair if you’re 
Pattern-Seeking 


The fallacy is clearer when you look at an extreme yet ac- 


curate analogy. 


I’m running an experiment to test whether a certain coin 
is biased. During one “trial” Ill flip the coin 10 times and 
count how often it comes up heads. 5 heads out of 10 
would suggest a fair coin; so would 6 or even 7, due to 


typical random variation. 


What if I get 10 heads in a row? Well a fair coin could ex- 
hibit that behavior, but it would be rare—a 1 in 1024 
event. So if my experiment consists of just one trial and I 


get 10 heads, the coin is suspicious. 


But suppose I did a “10-flip trial” one thousand times. A 
fair coin should still come up heads 3-7 times in each tri- 
al, but every once in a while it will come up 9 or 10 
times. Those events are rare, but I’m flipping so much 
that rare events will naturally occur. In fact, in 1000 
trials there’s a 62% chance that I’ll see 10 heads at least 


once. 


This is the crux of the fallacy. When an experiment pro- 
duces a result that is highly unlikely to have happened by 
chance, you conclude that something systematic is at 
work. But when you’re “seeking interesting results” in- 
stead of performing an experiment, highly unlikely 
events will necessarily happen, yet still you conclude 


something systematic is at work. 


Indeed, this is one reason why scaling a business is so 


hard: Scale makes rare things common. 


Bringing it home to marketing and sales 
data 


Let’s apply the general lesson of the coin-flipping experi- 
ment to Google Analytics. There’s a hundred ways to 
slice and dice data, so that’s what you do. If you compare 
enough variables enough ways, you'll find some 


correlations: 
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“Oh look, when we use landing page variation C along 
with AdWord text E our conversion rate is really high 
on Monday mornings.” 


Except you sound just like the baseball announcer, 
tumbling combinations of factors until something “signif- 


icant” falls out. 


Except you’re running 1000 coin-flip trials, looking 
only at the trial where it came up all heads and declaring 


the coin “biased.” 


Except you’re seeing streaks, hoping that this extra- 
high conversion rate is evidence of a systematic, control- 
lable force. 


The solution: Form a theory, test the 
theory 


You can use apparent patterns to form a theory; that’s 


good. 


The most exciting phrase in 
science isn’t “eureka,” but rather, 
“that’s funny.” 


—Isaac Asimov 


But then you have to test it, so that it’s science, not base- 


ball announcing. 
So: 


e Instead of using a thesaurus to generate 10 ad 
variants, decide what pain-points or language you 
think will grab potential customers and test that 
theory specifically. 

e Instead of rooting around Google Analytics hoping to 
find a combination of factors with a good conversion 
rate, decide beforehand which conversion rates are 
important for which cohorts, then measure and track 


those only. 


e Instead of asking customers leading questions or 
collecting scattered thoughts, form hypotheses and 
intentionally test them. 

e Instead of blindly following the startup founder who 
dramatically succeeded (the 1-in-1000 coin flip?), 
gather advice and observations that align with your 


style and goals. 


And then never stop testing your theories, because you 
never know when the environment changes, or you 


change, or you find something even better. 


There are two possible outcomes: 
if the result confirms the 
hypothesis, then you’ve made a 
measurement. If the result is 
contrary to the hypothesis, then 
you’ve made a discovery.” 


—Enrico Fermi 
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Reputation isn’t as powerful as you imagine 


by Jason Cohen on October 25, 2010 


Blogger with tens of thousands of subscribers launches a 
new venture... and gets only 2 signups. Not the advantage 
you thought it was. 


The most common reaction to my recent announcement? 
of starting a new WordPress hosting company was that 


this blog provides me with a ridiculous unfair advantage. 


1 Editor’s Note: This was written in 2010. 


But did it? 


“The fame is fine, and the money’s great, but 
sometimes I miss the plumbing.” 


Here’s what people said: 


“TYou’re starting from nothing] except approximately 
18,000 prospects. How convenient. ;) I wish I had that 


kind of mailing list starting out.” 


“You’re doing this the easy way, publishing this post 
so that thousands of users see it.” 


“It is, of course, simple to talk about how easy it is to 
be popular, when you’re the already established prom 


queen.” 


Fair point, but what actually happened after that post an- 


nouncement? How unfair was this advantage? 


Interestingly, Eric Sink got the same reception years ago 
when he launched a little company of his own. It’s worth 
hearing Eric defend himself because it’s just like my sce- 
nario, but because this happened six years ago I can re- 


veal his results at the end of this post: 


“Reactions to my Winnable Solitaire experiment were 
mostly positive, but several people claimed my experi- 
ment was “unfair” or “invalid”. In a nutshell, they ar- 
gued that because I am already “famous” for my writ- 
ings about the business of software, I have an advan- 
tage that is not available to my readers. My experiment 
is therefore meaningless because I did not duplicate the 
conditions a regular person would be facing when try- 
ing to launch their own micro-ISV.” 


Let’s start with the results of my announcement: 


WPEngine got two new signups. Only two. That with 
18,000 wonderful, loyal, friendly, supportive RSS sub- 
scribers and as many page-hits from Twitter and 
HackerNews. 


Not exactly the massive boost you nor I was expect- 
ing. I figured on 10-20 new customers at minimum and 


dreamed of 50. I was wrong by an order of magnitude 
2 Another example of why you cannot predict the future. 


Eric had a similar result: One month into his Winnable 
Solitaire experiment he had sold _a total of six copies. 


Hooray for fame. 


Let’s put this into broader context: At WPEngine we had 
50 paying customers (not prospects) before my post went 
live. Most are paying $49/mo, a few pay north of 
$1000/mo (large blogs with serious traffic). So whatever 
we did without the advantage of this blog was far more 
important, at least for getting initial customers. (Ill ex- 


plain exactly how we did that in future posts?) 
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3 Editor’s note: It took 13 years, but I finally wrote up that 
process for customer discovery, as well as a complete roadmap 
for Product/Market Fit. 


Still, the blog was instrumental in getting those first 50 
customers, but not because I’m able to push WordPress 
hosting onto 18,000 unsuspecting victims, a.k.a. 1000 
“true fans.” The blog did help with building the team. 


It’s no secret that the team is a critical factor in a start- 
up’s success; have you ever heard otherwise? But there’s 
precious little advice about finding and gathering that 
stellar team. Interviews on the subject invariably turn up 
explanations like “we went to school together” or “we 


“ 


worked together” or “we met at StartupWeekend " In 
short, you put yourself in an environment where you’re 
likely to interact with other intelligent, capable people, 
and hope that you find someone socially compatible who 
is also crazy enough to want to do a startup. It’s a good 


strategy, and anyway what else can you do? 


I knew I needed a killer team for WPEngine—not just 
“capable,” but a group that would itself be an unfair 
advantage. See, “WordPress Hosting” is already a com- 
modity, with every hosting company on Earth offering 
something at every price point from $0/mo, $5/mo, 
$15/mo, $40/mo, and even $500/mo + $200/hr con- 


sulting fees. In a mature market you need severe 


points of differentiation or moats, and one of those (I 
felt) had to be the team itself. 


We needed someone like Aaron Brazell. Aaron is a 
WordPress core contributor and the author of WordPress 
Bible (Wiley). He’s famous enough that strangers at 
WordPress conventions ask for autographs of their dog- 
eared copy of his infamous tome. He has seventeen zil- 
lion Twitter and blog followers, most of whom are them- 
selves active in the WordPress community. He knows all 
the major players in the industry including the key folks 
at WordPress.com, BZ Media (the CopyBlogger media 
group), ProBlogger, and members of the press at 
Mashable, TechCrunch, and others. 


Maybe with an Aaron we’d have a chance. His network 
should provide an ocean of free leads. His reputation 
transferred to the company would bless us with instant 
credibility. His press connections should give us pops of 
traffic and external legitimacy. His intimate knowledge of 
WordPress internals and roadmap should mean our ser- 
vice is technically superior. That’s a lot of advantages! 
Maybe enough to make or break a little new upstart. 


Well we got Aaron, and it’s because of this blog. When 
I called Aaron he was charging an obscene (and well-de- 
served) hourly rate for WordPress consulting in 
Washington DC, but he was yearning for the startup life. 
He was ready for the trade-off of definitely less money 
now in exchange for possibly more money later, and for 
building something of lasting value instead of the imper- 
manent drudgery of un-screwing hacked WordPress 


installations. 


And the blog sealed the deal. Aaron could have joined 
(or started-up) any number of startups, but he liked 
WPEngine because he wanted to do a startup with me. 
And he wanted to do a startup with me because the blog 
revealed my attitude, perspective, and credentials. 


Aaron picked up, moved to Austin, and has already been 
instrumental to our success thus far. 


So fame does help in important ways—enough even to 
deserve the title of “unfair advantage"—but startups are 
still hard and unlikely to succeed no matter who’s at the 


helm. Case in point? Eric Sink. 


Eric’s experiment eventually failed. Well, “fail” is a too 
harsh a word, it’s just the one in-vogue nowadays, espe- 
cially when describing an wonderful experience in which 
you had fun, learned a lot, grew as a person, and 
wouldn’t trade it for anything, particularly not a dull, 
predictable day job. You know, the kind of “fail” that 


characterizes a lot of software startups. 


On sales of $216, Eric sold Winnable Solitaire for a small 
sum. Of course neither the exit nor the to-date revenue 


amounts to anything that anyone would declare a 
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SUCCESS. 4 Editors Note: We went on to become a Unicorn, and in 2023 
are still growing and profitable. 


WPEngine’s revenue to date is several orders of magni- 
tude more, so hopefully we'll avoid that fate* Still, our But then, if it were easy it wouldn’t be worthwhile, right? 
expenses are also orders of magnitude more than Eric’s, 


and as I hope I’ve shown, although we have decided ad- 
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Color Wheels are wrong? How color vision actually works 


by Jason Cohen on January 31, 2011 


Artists say all colors are a mixture of red, yellow, and blue. Continuing this process produces the full color wheel 


But physics and TV screens and printers disagree. How you might have learned in school; a pretty, symmetrical, 


does color really work? 
satisfying device in which each hue melds seamlessly and 


Why are artists special? i a 


Ask any artist to explain how color works, and they'll 


launch into a treatise about how the Three Primary RED- ae RED 
; PURPLE ~ORANGE 
Colors—tred, blue, and yellow—form a color wheel: (TERTIARY) (TERTIARY) 


ANGE 


PURPLE 
(SECONDARY) 


(SECONDARY) 


the. 
baete 


YELLOW- 


BLUE- 


rena / res 
BLUE YELLOW 
(PRIMARY) (PRIMARY) 


BLUE- YELLOW- 
GREEN 
(SECONDARY) (TERTIARY) 


ier (TERTIARY) 
Why “wheel?” All other colors are created by mixing 


these three colors various proportions, they'll explain. In 
particular, mixing equal quantities of each pair of 


Primary Colors produces the Secondary Colors (orange, . 
Unfortunately, this crumbles under even 


green, and purple): 
minor scrutiny 


For example, open up your desktop printer and you'll see 


something quite different: 
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Three colors of ink which, when combined, produce all 
others: cyan, magenta, and yellow. (Black is included as a 
money-saver—black is the cheapest and most common 
color; it’s cheaper to have a black cartridge than to dump 
ink from the other three.) 


But wait! I thought the “Primary” colors were red, blue, 
and yellow, not cyan (bluish-green), magenta (bluish- 
red), and yellow. So on primary is the same (yellow) but 
the other two are different... yet these still generate color 
wheels containing all the other colors. So what does the 
“Primary” designation really mean? 


Also it’s not as simple as saying “any three colors can pro- 
duce all the others” because that’s clearly not true (by ex- 
periment). And it’s not as simple as saying “any three col- 
ors will do, they just have to be equally spaced around 
the color wheel,” because yellow is common to both the 
painter’s and printer’s wheel, yet the other two primaries 
differ completely (red and blue are primary in the 


painter’s wheel but secondary in the printer’s wheel.) 


TVs and computers are different yet again. If you stand 
close to a CRT (non-flat-screen), you can see that every 
pixel (or “dot”) is really three tightly-packed colored 


phosphors: red, green, and blue. 
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If you’ve done computer graphics you’ve been forced to 
name colors using these “RGB color values;” true geeks 
automatically think “yellow” when they see #FFFFOO. (If 
it's intuitive to you that #A33F17 is burnt orange, it’s 


time for you to leave the monastery.) 


This leads to yet another system of three “Primary” 
colors generating all the others, and yet another color 
wheel. This one is a little easier to explain—ink and paint 
are “subtractive” (adding cyan, magenta, and yellow 
yields black) whereas colored light is “additive” (adding 
red, green, and blue yields white): 


Still, we have yet another color wheel in which two (but 
not all three!) “primaries” match those of the artist’s 


wheel and none match the printer’s wheel. 


This isn’t adding up. Let’s turn to science. 


Physics makes it worse 


Physics is clear and certain. Light is a wave of electro- 
magnetic energy (and/or a particle, but for today it’s just 
a wave OK?) and, like a vibrating guitar string, light 
waves wiggle at certain frequencies. Some of those fre- 
quencies we detect with our eyes, and the frequency de- 


termines its color: 


~ Energy increases 


Short wavelength Long wavelength 
ene 


ATA AVAVAVAVAU ACN 


10nm 10°nm Tnm 103nm 10° nm Im 10°m 
1 1 1 1 1 t 4 


Gamma rays X rays Ultraviolet Infrared Microwaves Radio waves 
T T 


T T T T T T T T 
10%Hz 107Hz 10"Hz 10'Hz 10'%Hz 10"7Hz 10" Hz 10° Hz 10°Hz 10*Hz 107 Hz 
High frequency Low frequency 


Pail Visible light 
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Now we're getting somewhere! Or are we? 


First off, we’ve suddenly lost the notion of a “wheel.” As 
much as the previous color systems have contradicted 
each other, at least they all agreed that hues transform 
smoothly and continuously, one to the next, a beautiful 


symmetry with neither beginning nor end. 


But here we have a clear beginning (red) and end 
(violet). The colors in-between are continuous—and 
seem to generally match the order seen in the various 
color wheels—but then it just terminates with violet. 
How does it get back to red? What about that fuchsia / 
magenta / purplish-reddish color which is clearly 
present in every color wheel but missing from the 


physical spectrum? 
How can a color be missing? Where does it come from? 


But wait, we’re not done being confused. 


And another thing: Opposites 


Every seven-year-old kid in America is taught that “the 
opposite of red is green” and “the opposite of blue is yel- 


low.” But what does that mean exactly? 


After all, there’s nothing in that linear physical light spec- 
trum to indicate that any color is “the opposite” of any 
other, particularly not those two pairs. And the color 
wheels aren’t much help either; trying to match the “op- 
posites” on the painter’s wheel yields an unsatisfying 
asymmetry where two of the primaries are opposite, and 


the third is opposite from a secondary: 
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But “opposites” are real. In the early 1800s Goethe (yes, 
the Goethe) noticed that red/green and blue/yellow were 
never perceived together, in the sense that no color could 
be described as a combination of those pairs. No color 
could be described as “reddish green.” If you are asked to 
imagine “a green with a bit of red,” nothing comes to 
mind. In the following 150 years, various experiments 
tested this idea, all of which validated his observation. 


There’s something to this. Something neither the wheels 


nor the spectrum can explain. 


It’s time to get down to the real source of color: The 


ridiculous complexity of human beings. 


The answer: Physiology (of course) 


Caveat Emptor: The following is a gross and irresponsible 
over-simplification of what actually happens. But it’s cor- 
rect in its general thrust, and few people on Earth (myself 
excluded) are qualified to explain with complete accuracy, 
so in the interest of general illumination, no pun intended, 


OK maybe intended just a little bit, I’m doing it anyway. 


Of course it starts in the eye, where three types of cells 
called “cones” measure the amount of red, green, and 
blue light hitting the retina. 


“Ah ha,” I can hear you CSS freaks scream, “it’s RGB after 
all! I was right! All that time spent—nay invested—in re- 
membering that #001067 is the default title-bar color in 


? 


Windows 95 was well worth it! 


Hold on there, cowboy. Actually “amount of red, green, 
and blue” is a gross simplification (as warned!). Peaking 
under the hood (just a tad), the three types of cones are 
in fact denoted S, M, and L for “short, medium, and long” 
wavelengths, and each respond at different levels in a 


range of wavelengths: 


BN 


Normalized cone response (linear energy) 


400 500550. 600.~-650~-~-700 


Wavelength (nm) 


But I digress, and besides I did promise to be all gross 


and irresponsible, so let’s go back to that. 


So there are R, G, and B cones. The signals from these 
cones don’t go straight to the brain; they first pass 
through a pre-processing filter, and it’s this filter that 
explains all the mysteries. Actually there are three 


filters. 


Filter #1 works like this: 
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(to brain) 


Explanation: The more R there is, the more positive the 
signal; the more G, the more negative the signal. If 
there’s relatively equal amounts of R and G—whether 
from none of both, a little of both, or a lot of both—the 


signal is zero. 
This explains why there’s no “greenish-red.” Because: 


Let’s say R and G can go between 0 and 100 units of in- 
tensity. Consider the case of “full red with a little green,” 
where R=100 (full intensity) and G=25 (one-quarter in- 
tensity). Then separately consider the case of “strong red 


with no green,” where R=75 and G=0. 


In both cases, Filter #1 computes the same output signal: 
75. But remember the brain doesn’t get the raw R and G 
signals—it only gets the filter’s output—so the brain can- 


not tell the difference between these two scenarios. 


So there’s no such thing as “red with a little green’— 
there’s just a less intense red. The brain physically cannot 


see “greenish-red” because the filter removes that 


polarity. 


Knowing that blue/yellow is the other opposite pair, you 
can probably guess what Filter #2 is: 


(to brain) 


Here blue (B) is opposed with a combination of both the 
R and G channels. The R and G cones are stimulated ei- 
ther when there’s literally both red and green light (like 
when a CSS coder turns on both red and green as to cre- 
ate yellow), or when 570nm light (yellow, on the visible 


spectrum) stimulates both R and G cones. 


Filter #3 is simple: 


(to brain) 


In short, it measures the quantity of light without regard 
to hue. This is “how bright,” or “luminance” in color-the- 


ory parlance. 
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And magenta? It comes from full R and B with no G, ac- 
tivating Filter #1 full-positive, Filter #2 at zero. It’s not a 
physical wavelength of color, it’s just a combination of 


outputs of two filters. 


A corrected color wheel 


To do this “wheel” thing properly, you should represent 
the red/green and blue/yellow opposites. It’s not at all 


difficult, so it amazes me how rarely it’s seen or taught: 


@ 


Cale” 


Four primary colors? Yes, why not? It’s the closest thing 


to the physiology without getting complex. 


Bonus Brain Bender: The context / color 
connection 


This is just the beginning of color theory. To give you a 


glimpse at how complex it gets, consider this: 


When a color is juxtaposed to other colors, we perceive it 
as a different color. For example, most people will say the 
small square on the left is brown, whereas the one on the 


right is orange: 


Josef Albers, Folder IV-1 


Actually, the squares are exactly the same color! The sur- 
rounding context dictates the perceived color, on top of all 
that wavelength-physiology we just did. 


This makes sense because the brain projects abstract 
things it knows about the natural world onto your per- 
ception of color. For example, we know intuitively that 
shadows artificially darken colors, so our brains automat- 
ically account for this in our perception of those colors. 
(it’s called “color constancy.”) For example, you know 
that the dark and light colors on this hot air balloon are 


“the same:” 
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But it also results in optical illusions so powerful that 
even when you know the trick you still can’t see it 


correctly. 


Like this: Which square is darker: A or B? 


In fact A and B are the same color (#787878), but you 
can’t see it even when you know this. To prove it to my- 
self I had to open this picture in an image editor and ac- 
tually move one square over another to see it was the 


same. 
Freaky. 


Further Reading 
You got this far? You still care? Sheesh, you’re as weird as 


me. 


If you really want to lose a few days of your life, this is 
an amazing, in-depth treatise on color theory. That link is 
just page 1 of 8. Good luck. 


Printed from: A Smart Bear 
https://longform.asmartbear.com/color-wheels/ 
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The unfortunate math behind consulting companies 


by Jason Cohen on February 7, 2011 


Hiring even one person creates a lot less profit than you’d 
think, and launching a product rarely works out. Here are 
some tips for consulting companies. 
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“I dunno, lately I’m just not incentivized...” 


Consulting can be a great way to fund a startup or make 
a bunch of cash. It’s easy to start; Just pick an hourly rate 
and jump in. There are tons of online tools for time- 


tracking, project-management, and billing. 


But someday soon you'll notice there’s only so many bill- 
able hours in the day, and you'll be tempted to expand. 
Maybe hire an employee for $30 per hour and re-bill 
them at $60. Easy money, right? 


Unfortunately the math doesn’t work that way. 


Here’s what really happens, and a few ways to combat it. 


Double is nothing 


Suppose you hire an employee at $60,000/year. There 
are 2,000 working hours in a year—40 hours per week 
times 50 weeks—leaving two weeks vacation. So your 


cost per billable hour is: 


Nominal Cost: $30/hour 


In terms of working days (WD), there’s 250 in a 50-week 
year. In America we have ten federal holidays bringing 
the annual WD count to 240. Recomputing the cost of 


the remaining billable hours, we have: 
20 WD/month, $31/hour 


We also have employment tax in America. Rules are com- 
plicated and vary by state, but as a rule of thumb you pay 
15% in taxes including Medicare and Social Security. 
That changes the annual cost of your employee from 
$60,000 to $69,000 with an associated change in hourly 


cost: 
20 WD/month, $36/hour 


We've assumed an 8-hour work day, but any owner of a 
consulting company will tell you this rarely happens. Oh 
sure, founders work 60-80-hour weeks, but not employ- 
ees. Besides, even if you're in the office 8 hours a day it’s 
hard to be heads-down-on-task the entire time, and con- 
sulting is about billable hours. It’s hard to be billable for 8 
hours a day, every day. From what I’ve seen the weekly 
billables are more like 36 hours instead of 40. That 
means you’re short 16 hours per month, which is equiva- 
lent to missing two full (8-hour) days. With this loss, 


now we're at: 
18 WD/month, $40/hour 


Then personal life intrudes. You come in at noon because 
you had a morning dentist appointment. You take off ear- 
ly to wait for the A/C repair guy. You’re sick one day. All 
very reasonable and it doesn’t sound like much, but two 


half-days and a sick day means another two days lost: 


16 WD/month, $45/hour 
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We've assumed you're able to keep your employee com- 
pletely busy throughout the year without down-time be- 
tween projects, but that’s unrealistic. If you get two-week 
gigs, maybe there’s a day between them where the timing 
didn’t quite work out or you did some post-contract 
clean-up. If you land a six-month gig, that’s better, except 
it’s hard to time large deals like that to start and end ex- 
actly when you want, months in advance; in fact it’s com- 
mon to have 2-3 weeks of down-time between such 
things. Either way you lose at least 10% of your time, 
and that’s assuming a healthy incoming project pipeline, 
so knock off another two days (10% of the 20-day work- 


month): 
14 WD/month, $51/hour 


And let’s suppose you want to allocate just a little time 
for career development. After all, if you expect to hire 
and retain great people, they need time to keep learning, 
stay in front of the latest technology, and just have fun. 
Maybe that means going to a conference or two, or work- 
ing on an open-source project, or learning some new tech 
that’s needed for work. You see what’s coming of course: 
That’s another 10%: 


12 WD/month, $60/hour 


The true cost of an employee is double the nominal 
cost. If you bill out a so-called “$30/hour” employee at 
$60/hour, you'll only break even. You really need to bill 
out at $100/hour to make any kind of profit. 


Which is hard, because the client you're billing knows 
this person doesn’t cost even close to $100/hour. And 
when that client thinks about what’s “fair,” they won’t go 
through the computation I just did; they'll base it on the 
person’s nominal rate plus a little profit for you. This 
caps the amount you can actually re-bill before client 


feels ripped off. 


I hear you say: “Yeah, but at $100/hour that’s $40/hour 
profit, or $80,000/year to me! So that’s still really good.” 
But consider this: If your underling can be billed at 
$100/hour, can’t you bill at $150/hour at least? And if 


you worked just 20 hours/week (on average) at that rate 
you’d clear $150,000 annually without the hassle of hir- 


ing and managing an employee. Isn’t that better? 


What to do? 


So how do you mitigate these issues? Lots of ways, few 
of which are compelling, but all of which are quite do- 
able: 


Scale 
If you had five of these employees you'd be clearing 
$300,000 per year, which sounds better. Except not be- 


cause scaling creates more time and expense: 


e The client pipeline is much harder to maintain. 
Keeping 3-6 projects going year-round is a full-time 
job in itself—a job resting on your shoulders. 

e Employee turnover becomes more common, so you’re 
permanently in hiring mode. That in itself is time- 
consuming and expensive, and it’s hard to coordinate 
new hires exactly when other employees are leaving. 

e With six people you'll need office space with the 
attendant expenses, or a significant, possibly 
international travel budget for quarterly get- 
togethers. 

e With all these new tasks, there’s no way you can also 
manage the projects and client relationships and 
internal product development, so you'll need a 
project manager or a sales person or an office 
manager or some kind of help, and all of those come 
out of your profits. 

e None of these new tasks are fun or creative. It’s 
drudgery, and it’s all on you. Congrats, you’re a 


business owner. 


With these new expenses it’s not unusual to see that so- 
called $300k in profits came back to $150k. And now 


you're doing things you hate doing. Scaling is hard. 
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Charge more 
Any amount more you can charge goes straight to prof- 
its, so even charging a wee bit more (or a lot more) 


makes a big difference at the end of the year. 


Often, though, charging more pushes away your existing 
client base, and you'll have to find a new breed of folks 
willing to spend the big bucks. These are generally big- 
ger, more dysfunctional companies with 47-page Master 
Service Agreements and complex, ever-changing require- 
ments (though without an acceptance of “agile” or 
“lean”). It’s a living, it’s just a different living than being 
a Cool Cat HTML/CSS whiz doing fun projects for other 


entrepreneurs. 


This sword cuts both ways—even little discounts crater 
profitability. For example it’s common practice to give a 
10% discount if the client pre-pays, but although it’s 
great to have cash up-front, that discount comes right off 
your bottom line. (A better idea is to raise your prices by 
P%, then offer a P% discount for pre-pay. Then your 
cash-rich customers are paying the same, but others are 
creating more profit in exchange for delayed payment—a 


reasonable trade for both parties.) 


Bill more hours 
Billing even 5 hours more per week significantly increas- 


es profitability. 


The problem is convincing your employees to work more, 
because to them that’s “overtime” on a fixed salary. 
What’s in it for them? 


One answer: Split overtime billings with them. It’s pure 
profit to you and your employee can earn a significant 


bonus. 


Build a product 
Many consulting companies have a few ideas or even 
working internal projects which they hope to someday 


turn into money-making products. The usual story is: 


We built this product because we needed it ourselves. It 
gives us an advantage in consulting because it acceler- 
ates our development and enhances our sales pitch. 


But surely we’re not the only software developers on 
Earth who would find this tool useful, so whenever we 
have a lull in client work we work on our product. 
Someday that will cash-flow by itself, at much better 
margins. 


Indeed there are conspicuous companies who grew suc- 
cessful product lines in exactly this manner: 37signals, 
FogCreek, and Pivotal Labs. Unfortunately, for each of 
those there are hundreds if not thousands who toil away 
at pet projects which never see the light of day. Some 


reasons: 


e Converting an internal project to a user-friendly one 
requires drudgery: Onboarding flows, 
documentation, intuitive user interface, installers, 
password-reset, and fixing those 200 bugs that 
internally you’ve just learned to work around. No one 
wants to do it, and it’s no one’s job to do it, so it 
doesn’t happen. 

e It’s no one’s job to market or sell the product. No one 
has the time, and it’s quite possible that in a roomful 
of consultants no one has the skill-set either. So you 
“put it out there” hoping that the magical world of 
“social media word of mouth marketing” will bestow 
eager users upon your sign-up form. But, of course, 
that’s not what happens. 

¢ You have five kinda-cool projects, any of which might 
be awesome if given enough TLC, but because your 
attention is spread among all of them (and even then 
only in fits and spurts between client work) none 
gets to the stage of being usable by anyone else. Lack 
of focus kills. 

e Billable hours trump product development; it’s 
always second-class, so you get a second-class side- 
project. 

e Thinking that just because the product is “great for 
us,” that there’s a market out there. Usually there’s 


not. (Here’s how to figure it out). 
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Most of the problems above can be fixed by treating the 


product as a client: 


¢ The product is listed as a client just like all the rest. 

e The product is given a budget, even to the point of a 
separate bank account (and corporation?) so it’s 
clear how much you’ve spent on it, and the project 
pays consulting fees (perhaps at cost instead of full- 
rate). 

e Hours are scheduled and prioritized like other 
clients, and doesn’t slip just because it’s not a “real 
client.” 

e Ifthe (potential?) income from the project cannot 
justify the cost, you might consider it a loss and stop 
completely rather than continue to limp along as a 


money-losing project. 


Use subcontractors instead of employees 

If you pay your underlings for exactly the number of 
hours you rebill them, you avoid all of the issues above: 
No employment taxes, no worries about 36-hour weeks 


or vacations or project gaps. It’s simple time-arbitrage. 


Terrific! Unfortunately, subcontractors charge a lot by 
the hour—a lot more than employees. Of course they 


do... exactly for the reasons above! 


You haven’t eliminated those effects, you’ve just moved 
them to someone else’s profit-and-loss report. If you 
hired that same employee as a consultant she would 
charge you $60/hour instead of $30/hour and your prof- 


its are still the same. 


Still, there’s a lot less headache using consultants, so this 
might still be worthwhile. Simpler taxes (in America), no 
bargaining for vacation time, no prodding them to bill 45 
hours this week to make up for last week. Even from a 
psychological standpoint this might be better: It’s easier 
to give up $60 when you charge $100, but it causes some 
consternation to pay someone for a “day’s work” when 
there’s no work to do. 


Is consulting really that bad? 


Consulting is a great way to earn a living and a smart 


way to self-finance a startup. 


The trick is to avoid all these traps. For example, you 
know billing a full 40 hours per week is critical, so make 
it a habit to review hours weekly to make sure no one is 
consistently falling behind. As another example, set up 


incentives where employees get to share in the profits. 


It’s always hard. Most consulting companies don’t make 
much profit, and it’s one in a thousand that has the disci- 
pline to launch a successful product during off-hours. If 
you’re going to make it happen, you yourself need to be 


serious, disciplined, and relentless. 
But you can do it. 


And if you do, you’ve just self-financed a startup, made a 
nice living, mitigated much of the risk of product-only 


startups, and built a great team in the process. 


Printed from: A Smart Bear 
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When you want to quit because it’s just not worth it 


by Jason Cohen on February 28, 2011 


Are you crying in the shower because you can’t handle it 
anymore? Beyond Impostor Syndrome: Complete melt- 
down? Well, at least you’re in good company. 
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I’ve been there too. It sucks. 


Most startups fail “only” because the founders stop work- 
ing on them, and often it’s because it’s emotionally drain- 
ing. I don’t care who you are or how strong your ego is, 
you will have these moments—perhaps a continuous 


stream of moments—when you can’t take it anymore. 


I cannot remember the number of times I was so over- 
whelmed at Smart Bear that I almost threw in the towel. 
Close the bank accounts, close the doors, turn off the 


website, bounce the email, and just... 
stop. 


Sounds dramatic, but it’s no exaggeration. You'll hit 
these walls too. I know you will because this is what peo- 


ple admit in little rooms with other people who truly un- 


derstand and won’t name them publicly, and also there’s 
a steady stream of people saying it out loud on Twitter, 


many times per day. 


Maybe a little commiseration will help you get through 
it. 


You expect these moments to happen at the beginning of 
startup life—when youre least confident, have the worst 
product, and the least knowledge about your customers 
and the market. Paul Graham and Trevor Blackwell cap- 
tured it pretty well: 


The Process acrid 
Buyer 
Acquisition 
Wearing Off of Liquidity 
eho of Novelty 
of Initiation Wiggles of 
False Hope 
The Promised 
Trough of Land! 
Sorrow \ 


) 


Releases a Crash of 
of Improvement Ineptitude 


The pain is not limited to the beginning of the ven- 
ture. It’s still there years later, despite real revenue, prof- 
itability, customers arriving every day, and a great team. 


Since that is not obvious, I'd like to share a personal 


story. 


Four years into Smart Bear I had several employees get- 
ting paid decently (which at a bootstrapped startup is 
hard to do!), a product that people were buying, and we 


were doing around a-million-a-year. Life was good! 
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I was negotiating my first true “Enterprise sale!” This 
was going to be our biggest order to date—something 
like $200,000. (Yes, one-time revenue. It was a simpler 
time.) Actually “negotiating” is the wrong word because I 
don’t believe in price negotiation, even with enterprise 
sales (an area that most people claim must include auto- 


matic discounting.) 


1 If you aren’t familiar with the professional mosh-pit that is en- 
terprise sales, here’s a primer. 


The person with whom I was negotiating wasn’t the end 
user, nor the boss, nor the boss’s boss, nor anyone in that 
chain of command. See, big companies have entire de- 
partments devoted to dealing with vendors like you and 
me, and when it comes to negotiating, these departments 
harbor terrorists with titles like “Procurement Manager” 


or “Strategic Sourcing Manager.” 


I say “terrorists” because they use fear tactics to get their 
way, yet they have no power other than fear. Imagine the 
worst stereotype of a salesman, except instead of selling 


you something, their job is beat a discount out of you. 


Now to be fair, many vendors do take advantage of large 
companies—overcharging (because “They can afford 
it!”), or promising one thing to the users and sneaking 


something else into the invoice. 


But mostly it’s because of the traditional enterprise sales 
dance, reminiscent of the lumbering mating dance of the 
great blue whale. The vendor asks for too much money; 
the client is astonished at the price. Both calmly explain 
that this is a deal-breaker. Then the vendor capitulates 
30% but only if the client signs a three year maintenance 


contract (which they wanted anyway). The deal is struck. 


This tradition continues because of perverse, wasteful in- 
centives. The vendor’s salesman likes this because some- 
times she gets away with a high price which pads her 
commission check. The Procurement Manager likes this 
because he can show his superiors how much money he’s 


“saved” the company. 


So big companies need a Defender of Evil Vendors, I get 
that. But that’s not enough for these guys; it feels to me 


like an attack, not a parry. 
This is how the conversation went: 


PM: What kind of discount are you offering? 

Me: We don’t discount; instead we put our pricing on 
our website so there’s no misunderstanding. 

PM: Well I’m going to need some kind of discount. How 
about 30%? 

Me: As it says on our website, we don’t discount. 

PM: But I’m buying 400 seats! 

Me: Yes, and we already provide a nice discount for 
bulk orders, which is already included on the invoice 
and documented on the website. 

PM: You don’t understand, I always get a discount. I’ve 
done business with 47 other vendors and all of them 
give me at least 20% off. 

Me: There’s always a first! 


So far it’s actually OK—I’m the one refusing to plod 
through the mating ceremony, wanting to skip right to 


the wedding night. I expect push-back. 


Here’s where it gets nasty. I remember sitting there on 
the phone getting lambasted for my intolerable ignorance 
about the Way It Works. I was told, and I quote, “You 
have no business selling anything to anyone.” My ob- 
stinate ignorance is a deal-breaker for the entire sale be- 
cause of what it implies about my company in general— 
after all if I don’t even understand the purchasing process 
there’s no chance in hell my software’s going to work! 
Furthermore, despite my ignorance, I’m unwilling to lis- 
ten to the rules, unwilling to learn, which means there’s 


no hope for me. 


Pll never forget how this ended: 


PM: OK that’s it, you give me no choice. I absolutely 
cannot approve this deal, and furthermore I’m recom- 
mending that we never work with your company in 
any capacity. At this point, even if you gave me a dis- 
count I would still reject it. 


Here’s where I’m supposed to unleash my intellectual for- 
titude. I won’t capitulate, will I? I won’t let this guy insult 


and bully me, will I? C’mon, I’m the strong-willed confi- 
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dent entrepreneur with the stoic well-argued voice of 
reason, whereas he’s the sleezeball with the tedious day- 
job—surely ll laugh as his words roll off me like water 
off an oiled duck’s back. 


Just the opposite. I felt like throwing up. He’s right, who 
do I think I am? I’m a baby geek playing in the adult’s 
house and I don’t know what the hell I’m doing. I have 
these naive ideas about how the world should work and 
how people should treat one another, and it’s all just in- 
credibly silly and ignorant. And it shows. And now this 
guy is going back to those other folks at the company 
whom I actually like, and worked hard to earn their 
trust, as they laugh together over can-you-believe-how- 


dumb-he-is and we-can’t-possibly-do-business-with-them. 


It’s over. They've seen through me. It’s just a matter of 
time before others do too. That’s the end of deals like 


this. 


Why am I doing this anyway? This is supposed to be fun 
and fulfilling but at this moment as we say in Texas I feel 
like ten tons of shit in a two-ton bag. What I like is writ- 
ing code—why am I even trying to play this sales game? 
Why not just go get a job where I only worry about 
whether or not I can write code—because I sure as hell 
can do that—and let the natural salesmen do all this 


crap? 


Is the money worth it? What money, we’re still bootstrap- 
ping and I still don’t get a regular salary. Is the promise of 


money worth it? Worth these feelings of inadequacy? 


After days (yep, days) of fretting like this, it converted 
from despair to anger. Who the hell is this guy? Some 
asshole who isn’t good enough with money to be an ac- 
countant, not even slimy enough to sell cars, this guy 
whose only skill is to be a jerk, some guy who has never 
had to make payroll or take a risk or put himself out 
there, this schmuck is going to tell me I’m the one who 
isn’t good enough, I’m the one who has no business sell- 


ing software? 


Worst of all, ’m letting him make me feel like a pile of 


shit! 


Well if you’re waiting for the big moment where intellec- 
tual reasoning finally defeats weak, irrational emotions, 
I’m sorry to disappoint you, because that moment never 
came. I know it’s dumb and illogical, but there it is. It’s 
trivial and baseless but I still carry that experience in a 


corner of my thoughts. That’s how emotions work. 


By the way, this guy turned out to be totally full of shit. 
He had, in fact, no power to stop the deal. When I finally 
got my main buying contact from that company on a con- 
ference call with the PM, the conversation was this short, 


and as close to word-for-word as I can recall: 


My Guy: So, what’s holding up procurement’s ap- 
proval? 

PM: Nothing, just some paperwork, we'll have it done 
by Friday. 


All of that angst for nothing. Son of a bitch! 


Years later I was on site at this company and I finally met 


this guy face to face. 
I still felt small. 
Want to say I’m weak? Or he’s strong? 


Who cares, the point is: Getting through this slog of a 
thing that’s a startup—or anything difficult and worth- 
while—doesn’t require that you’re always confident or 
stoic or smart or right or wise. You don’t need to match 
the emotional stability you see from the big bloggers. 


(Which is mostly a facade anyway.) 


It’s not even about “overcoming.” Maybe you don’t over- 


come, you just get through. 


It’s about sticking through the tough parts, whatever 


your foibles and weaknesses. 


Living through it, not beating it. I never have, to this 
day, “beaten” that PM, not emotionally, not if ’'m being 


truly honest. 
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Welcome to your crucible. 


I’m not saying tenacity is all it takes. Just that without it, 
you'll stop. It’s so easy to stop. There’s so many reasons 


to stop. 


And that—stopping—is how most little startups actually 
fail. 


PS. Since writing this article in 2011, I was able to avoid 
burn-out and these other thoughts with a framework that 
I detail in this article from 2022. 
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Never say “no,” but rarely say “yes.” 


by Jason Cohen on April 25, 2011 


“Focus” requires saying “no” to most things, but there’s a 
way to do it that allows you to say “yes” exactly when it 
matters most. 


“I know he’s not much to look at, but the 
symbiosis is great!” 


Everyone says small startups require focus. Say “no” to 
anything that distracts from your goal, your vision, your 
strategy, tempting though it is to explore all opportuni- 
ties, hoping each time that this is the one that will cata- 


pult you to “success” (whatever that is). 


Lack of focus results in half-assed initiatives, each inter- 
rupted by apparently greener pastures before you’ve in- 
vested the time and devotion it deserves. Learn to say 


“no!” 


Ah, but then again you must also experiment with new 
ideas. Fail fast! Pivot! Test! Doubt! Always be collecting 
evidence that you’re wrong, always be trying new things 
in case you’ve been blind. Never pass up an opportunity 


to change, learn, grow. 


So... how are you supposed to explore other ideas if 
you're also supposed to be saying “no” to anything that 


diverges from The Plan? 


Here’s what I do: I never say “no.” But I carefully 


qualify “yes.” 


I learned this trick in high school. In the mid ’90s it was 
clear that Apple had lost the personal computer battle 
and all their developers were fleeing like rats off a sink- 
ing ship into the ocean of opportunity that was Windows 
95. As a maven of the Macintosh API and still willing to 
admit it, I landed small contracting jobs fixing up code 
that other developers wouldn’t touch. 


My typical rate was $25 per hour, which feels like a lot of 


money when you're 17 in the ’90s. 


One day I got a call from some poor schleps I didn’t want 
to help. They had just completed a new product written 
in Java and it was broken on a Mac, and could I help? 
None of their customers used Macs, so they didn’t think 
Macs were important, but then it turns out the main in- 
vestor is keen on seeing the demo on a Mac, and when 
they tried it, it didn’t work. (Yay investors! Yay supposed- 


ly-cross-platform-platforms-that-aren’t-really!) 


I wanted no part of this. Java was brand new and known 
to be full of bugs, and anyway I was a C/C++ kind of 
guy, and I didn’t want to get involved in an academic fad 
language like Java! 


1 So yeah, I simultaneously decided that (1) Java is a fad and 
(2) I’m sticking with the Macintosh Toolkit; five years later one 
those platforms had zero developers and the other had one 


million, and I picked exactly wrong. Predicting the future is 
hard. 


I could have said “no.” Given my specialty and my goals, 
traditional career (or startup) theory says I should have 
said “no.” 


But instead, on the advice of an older, wiser friend, I 
showed up at their office and said I’d do it for $100 per 


hour. 
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I fully expected them to laugh in my face. Maybe I would 
receive a condescending talking-to about the audacity— 
nay, the impudence!— of someone of my tender age and 
meager experience walking in here and demanding such 
outrageous compensation, someone who, let’s be clear, is 
technically too young to even enter into a legal consult- 


ing agreement in the first place. 


And then I would have slinked out of there, embarrassed 


but ultimately no worse for wear. 


But that’s not what happened. They looked me up and 
down, their faces painted with both incredulity and sur- 
render. They said OK. An hour and a half later, every- 
thing was working. The difference between saying “no” 
and getting $150 for about two hours of my life was all 
in how I phrased “yes.” 


And I have stories that went the other way, which are 


just as important. 


At WP Engine, for example, we’re constantly talking to 
large bloggers who want to move to our system. These 
are folks with big requirements—tens of millions of 
monthly page-views, traffic spikes, custom code, perfect 


up-time, and 24/7 support. 


Should we take on those clients? Maybe not—after all 
the stated goal of WP Engine is to serve the “middle mar- 
ket”—the folks who have outgrown free blogs, don’t like 
maintaining their own servers, but aren’t so large that 
they have extreme hosting demands. That’s our profitable 


sot ond. 
niche~. 


2 Editor’s note: This was written about 1 year into WP Engine’s 
life; in 2023 WP Engine is 13 years old with 200,000 cus- 
tomers, and powers more large WordPress sites than anyone on 
Earth. This was a great segment to start off with, but we 
evolved as we scaled. 


Or so we think. But if we just say “no” to these big blog- 
gers, maybe we're closing the door on big, important or- 


ders. Perhaps the entire company should pivot—maybe 


it's easier or more profitable to serve 100 large blogs 
than 1000 medium ones. But how do we know if we say 


“no?” 


Then again, if we say “yes” we might really be screwed. If 
we can’t provide them the human and technical service 
they expect, now we’ve hurt a blogger, we get bad press, 
and we’ve wasted a bunch of time. Or even worse, we 
hold on for dear life but it’s extremely unprofitable, and 
now we have this expensive, time-consuming albatross 


around our necks. 


So we've said “yes” by quoting high enough that we 
know for certain we will make good money on the deal, 
so much so that it will partially fund something else we 
want to do. Maybe that means a big new advertising 
campaign, or hiring another WordPress expert for our 


staff. 


There was one especially large customer where we liter- 
ally thought of it like this: This deal needs to be big 
enough to not only make a reasonable profit on the oper- 
ating expenses, but pay for an entire developer’s salary 
(assuming bootstrapped, put-in-elbow-grease-for-stock 
low salary), because we know this new customer will oc- 
cupy a lot of that person’s time, but all the remaining 
time we get “for free.” 


So we've given a lot of qualified yeses, and many were 
rejected. (PS. Now we're able to say “yes” to those same 
bloggers, but that’s because over time we’ve taken on 
bigger and bigger customers, and now a blogger with 30 


million month page-views is something we know we can 
handle.) 


At Smart Bear I used this principle yet again. Companies 
would fly me out to help them implement a peer code re- 
view process, which half the time actually meant that 
“management” wanted me to convince everyone else that 
code review was a good idea, and invent a process pain- 
less enough that they might actually do it. 
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From a business perspective, this was a poor use of my 
time. These folks had already bought our software, so it 
didn’t sell more seats. When you counted a travel day on 
either side of the engagement, the time I lost could have 
been spent landing just one additional customer or make 
some important changes to the code, either of which al- 


most certainly makes us more money. 


Therefore, initially I just said “no.” But again that’s 
wrong. Eventually I said “yes,” but the price was 
$2500/day including travel days, which for these sorts of 
engagements is unheard of. (Typically you get reim- 


bursed for travel expenses but not paid for that time.) 


This immediately cut out most trips, but some remained. 
On those trips I’d haul in $10,000 for a week of easy 
work, which I’d often combine with a long weekend with 
my wife. And anyway those people really wanted me 


there, which made the work that much more enjoyable. 


So the principle is straightforward: Set the conditions 


of “yes” such that: 


1. If they say “yes,” you’re happy because the terms 
or money are so good, it more than compensates for 
the distraction, perhaps funding the thing you really 
want to do. 

2. If they say “no,” you’re happy because it wasn’t a 
great fit anyway; it’s not a worthwhile return on your 


time and effort. 


So that’s the punch-line, but before you leave I’d like to 
over-emphasize the idea of “funding the thing you re- 


ally want to do.” 


This can take many forms, but it’s the single best way of 


figuring out how to qualify your “yes.” Examples: 


“Yes” if it pays for an entire additional person. 

e “Yes” if this extends the runway of our startup by at 
least three months. 

e “Yes” if it completely funds development we’d like to 
do anyway. 

e “Yes” if it means one of the co-founders can quit her 
day-job. 

e “Yes” if it will completely pay for three new 


marketing efforts. 


Think of it like another form of funding. Funding is al- 
ways a distraction from actually running your busi- 
ness, so the amount of money you get must be trans- 
formative to the business. Each of those bullet points 
are transformative, in that each has the potential to move 


your company from “hobby” to “real business.” 


And if they say “no,” you’re fine with that, because it 
would have been a distraction which wouldn’t have 


moved the needle. 
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Easy to criticize, hard to create 


by Jason Cohen on December 5, 2011 


It’s easy to explain why any given business will fail. So 
what? But neither is it wise to totally ignore the critics. 


—_ ANDETZSaN 


“By the way, your id 
says hi.” 


I can rip any business idea to shreds. 


Take Netflix! : The costs of inventory logistics, millions of 
non-technical customers, the Postal Service, and loss 
from wear and delivery will make profit impossible if you 
keep retail prices reasonable. Movie-watchers are accus- 
tomed to the immediate gratification of browsing and se- 
lecting. People will duplicate movies, pissing off the pub- 
lishers. Blockbuster will duplicate the model and under- 
cut the price, combining the convenience of home deliv- 
ery with the equally convenient option of store browsing 


and returns. 


1 Editor’s Note: This was written in 2011, before Netflix pivoted 
to streaming. 


Terrible idea that could never work. Except that now 
Blockbuster (who did copy it) is bankrupt and Netflix 
turned a profit of $60m last quarter, which is more than 


Blockbuster was doing at its peak. 


I suppose if your goal in life is to be right, you should bet 
against every startup, certainly against any novel ideas. 


You'll be right much more often than wrong. 


Right, but useless. Right, but ensuring that nothing new 
or great will be created. We can even show mathemati- 
cally why this is flawed, but you don’t need math to 


know that this isn’t a useful way to live. 


The goal of the entrepreneur is not to be “right.” At 
least, not right about everything, all the time, from the 
very start. 


It’s to construct with humility. To have strong opinions 
but seek better ones. To infect customers and employees 
with your peculiar disease. To live to fight another day. 


To acknowledge being wrong before the fault turns fatal. 


To realize that even if a particular venture is a “fail- 


” 


ure,” it never really is. I’ve never met a “failed” en- 
trepreneur who doesn’t proudly declare they’re better off 
now: developed skills they never thought possible, 
learned what makes them excited to get up in the morn- 
ing, came face-to-face with their fundamental limitations, 
and now more confident than ever in what they are capa- 


ble of, regardless of the future shape of their career. 


The Lean Startup movement? also values learning and 
experience over “being right,” specifically casting this 
concept as “running experiments.” And the great Francis 
Crick (Nobel-winner for discovery of DNA) posthumously 


agrees with that analogy: 


2 Editor’s Note: The Lean Startup book was published in the same 
year as this article. 
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The dangerous man is the one 
who has only one idea, because 
then he'll fight and die for it. The 
way real science goes is that you 
come up with lots of ideas, and 
most of them will be wrong.” 


—Francis Crick 


Real science is mostly being wrong, but staying vigilant 
and honest enough to continue being wrong until you’re 


not wrong. So it is with dating. Or startups. 


The problem with armchair criticism is it’s so easy. Of 


course it’s easy. 


Designers want to be “right,” but have to temper that 
with analytics. The true masters are those who embrace 
metrics as an incontrovertible goal while also being cre- 
ative enough to retain the aesthetics, creativity, and phi- 


losophy which makes their craft an art. 


Programmers want to be “right,” but have to temper that 
with getting things built and shipped, even before it’s 
ready, even with bugs, willing to build what customers 
actually want, and indeed features customers truly love 


instead of rationally calculated. 


Backseat advisors and pundits (like me) want to be 
“right,” but have to temper that with the more important 
goal of inspiring people to be a better version of them- 
selves instead of indoctrinating them with our own biases 


and experiences that contain unknown quantities of luck. 


As a founder, you still need to listen to critics with half an 
ear. Sometimes you have been blind, and they’re showing 
you how. I still resist opposition to my ideas, more than I 
know I should. I still hate being wrong. 


But it has to be possible to be able negate you, otherwise 
you are certainly pig-headed. As the founder of WP 
Engine I feel that most days are filled with more contra- 
dictions than construction—balancing customer needs 
with internal costs, following our nose against what com- 
petitors are doing versus staying true to our “vision” 
while also flexing as we learn what the market wants and 


even what we want for ourselves. 


It’s easy to criticize but it’s just as easy for you to dis- 
miss criticism without consideration. That’s equally 


hazardous. 


Whenever you get the criticism ask: Is there some truth 
behind this? Does it sting because there’s an element of 


truth that I don’t want to face? 


Ignore the messenger, ignore the emotion, and see 


whether it’s a gift of truth, poorly wrapped. 
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When being “first” is not a competitive advantage 


by Jason Cohen on December 19, 2011 


Is it good to be “first?” It seems so — what’s the point of 
being a copycat? While “first” sounds impressive, in reality it 
doesn’t predict who will win. 


@jasonlk 


& Jason + Be Kind + Lemkin ¥ @ 


Top advantages to being a first mover: 


- No roadmap 

- Tiny market 

- Customers aren’t sure 

- Limited legitimacy 

- Never budgeted 

- VCs don’t get it 

- Many VPs don’t get it 

- Tiny TAM at first 

- Seems minor to many 

- Why wasn’t it already done if it’s so important 


It’s great 


“We started the trend.” 


“Wherever you hear companies talking about Managed 


WordPress Hosting, they are following our lead.” 


This is how one of our competitors attempts to differenti- 


ate and compete with us at WP Engine + Is it effective? 


1 Editor’s note: This piece was written in 2011, when WP Engine 
was one and a half years old. As of this note in 2023, WP 
Engine is a Unicorn, while the competitor’s growth leveled off 
many years ago at less than 1/30th the size, then folded into a 
much larger company. With the benefit of hindsight, the point- 
of-view in this article was accurate. 


Many startups insist they're “first” at something. As they 
should—what’s the point of a new company that adds 
nothing new to the world? But is “first” a good thing? 


Does it make you better than your competition? 


On the surface being “first” sounds impressive, imply- 
ing innovation and leadership. But upon reflection, it’s 


not. 


Google wasn’t the first search engine, the iPod wasn’t the 
first MP3 player, DropBox wasn’t the first cloud filesys- 
tem, Excel wasn’t the first spreadsheet, Dell wasn’t the 
first PC manufacturer, Facebook wasn’t the first social 
network, and Cisco didn’t invent the router. Yet they are 
both the market leaders and their market’s greatest 


innovators. 


“Oog like first two, last one no get.” 


Being first does imply innovation, but also implies the 
faults of “version one.” The first electric car, while innov- 
ative, was pretty bad by most standards—short-range, 


sluggish, unattractive, and constantly in the repair shop 


2 Tesla solved the first three of these four qualities, further 
demonstrating the point. Counter-point: Tesla was the first to 
create good-enough batteries, first to realize that “perfor- 
mance” was non-negotiable, and first to figure out how to 
leverage the different engineering constraints of EV to create 
the safest sedan ever made. 


So too with software—‘“v1.0” might imply novelty, but it 
also implies “buggy” and “incomplete.” Who wants their 


business to depend on a v1.0 product? 


Even if you consider WP Engine to be nothing more than 
an imitator of this competitor (we’re not, but that’s a sto- 


ry for another time), the electric car analogy is apt, be- 
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cause in fact we’re an improvement in almost every way. 


We’re not the first electric car; instead we’re Tesla. 


This competitor had 40 minutes of downtime last month; 
we had 0 (according to Pingdom, an unaffiliated 3rd-par- 
ty). We’re 30% faster at delivering HTML (also 
Pingdom). Our security service is better—we guarantee 
cleanup of any hacks while this competitor simply claims 
they’ve never been hacked (though we’ve found malware 
in customers of theirs who switched to us). Our support 
is more responsive and knowledgable because we employ 
more WordPress experts per 1000 customers than 


anyone. 


Those are tangible competitive advantages; being 


“first” isn’t. 


In fact, being first can be a shackle because you’re stuck 
with supporting the legacy of that first version—features 
you thought were important but aren’t now (and maybe 
never were, a journey that new competitors will skip), 
grandfathered customers who are no longer profitable, 
architectural choices you wish you could change (and 
competitors started out avoiding), and a reputation that’s 
hard to revamp even if your company has in fact 


changed. 


Worse, being first means you made all the mistakes in 
public. New competitors get to swoop in afterwards with 
the hindsight that you created—in pricing®, positioning, 
marketing, features, design, ... everything. They get a 


running start without all your baggage. Thanks to you. 


3 Editor’s note: Years later, speaking to founders of a different 
competitor who didn’t yet exist at the time of original publish- 
ing, I asked how they determined their pricing. The answer: 
“We figured you’d already done the research and tested differ- 
ent things, so we just copied y’all.“ 


Of course there are situations where being first is indeed 
a massive advantage. This exception arises when you’re 
not only first, but also able to expand fast and continue 


innovate. Early MP3 players didn’t do this—they didn’t 


get huge traction and weren’t creative in hardware or de- 
sign, which meant Apple had the space to innovate in de- 


sign while the market was still largely up for grabs. 


But McDonalds and Ford were first and kept their leads, 
because they never stopped innovating in process and 
product, constantly driving down costs while adding new 
desirable features, using their lead to fund even stronger 


growth. 


Amazon is the quintessential example of “first, but relent- 
lessly innovative” in cloud infrastructure. They didn’t just 
put “cloud computing” on the map, they’ve never stopped 
lowering prices while expanding both depth and breadth 
of service. If they keep it up—and there’s currently no 
reason to think they won’t—it’s hard to see how anyone 
could overtake them. And so far no one has come close, 


though at least four multi-billion-dollar companies are 


trying* 


4 Fditor’s note: In 2023 this has not only held true, but more: 
They stopped lowering prices and still remain in the lead. Here 
is a case study with an explanation of the strategic moats that 
constructed this success. 


So yes it can work, and a well-funded startup combined 
with a stellar team and bit of good fortune can occasion- 
ally pull it off. But most little companies aren’t in that po- 


sition, and it’s often not the best risk/reward anyway. 


Being “first” isn’t an end in itself—it’s not an advan- 
tage, not a feature, not a benefit the customer can experi- 
ence. It can be a means to one of those ends—it can be 
levered into market dominance with persistence and the 
right strategy—or it can be a manacle that locks your 
company into a burden of legacy that allows newer com- 
petitors to pass you by. 


You should be proud of being “first” at something. But 
like being “disruptive,” being “first” isn’t necessarily 


desirable. 
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It’s just history. 
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“ROI” is the wrong way to sell your product 


by Jason Cohen on October 10, 2012 


Customers ask for ROI calculations to justify purchasing 
your software, but it still doesn’t convince them. Here’s what 
to do instead. 


Novemger 


"As you can see, we've not only kept up with the 
Jones, but surpassed them. Next up, the Nelsons." 


I can’t remember how many times at Smart Bear I tried 
to sell our main product Code Reviewer with the argu- 
ment that it “saves you money.” Some customers de- 
manded it in the form of an ROI spreadsheet. So we gave 


it to them. 


The argument makes sense (though it’s wrong). Code 
Reviewer is tool which helps software developers review 
each other’s work, just like an editor of a book. It cuts the 
time of a code review in half—maybe better—because it 
eliminates busywork. We even lead with that in our one- 
pager (because back in 2004, sales calls were in person, 


and one-page leave-behinds were useful): 


CodeReviewer” 


Cut peer code review time in half. 


Typical Peer Review 


CodeReviewer 


Find a developer to perform the review. Either 
drag her into your office or package up all 
changes (both source code and diffs) and ship 
them to her. It’s easy to miss a change. 

10 minutes 


Pick the reviewer from a list. Changes ate 
detected, packaged, and shipped automatically 
through version control integration. 

1 minute 


She reviews the changes. If she’s looking over 
your shoulder, you scribble down comments 
somewhere. If you've shipped the changes, she 
must record file/line numbers with comments. 
20 minutes 


She reviews changes in unique before/after side- 
by-side view. She clicks on a line to leave a 
comment. 

15 minutes 


You make corrections, manually locating the 
spot for each comment. If you want her to OK 
your corrections, you must tepackage and ship 
everything. She can’t tell what’s corrected and 
what’s the same as the last submission. 

25 minutes 


Total: 45 minutes. 

Problems: No audit trail. Insufficient 
tools for packaging and examining 
differences against whole soutce files. 
Hard to maintain and distribute a project 
checklist. 


You make corrections, jumping directly from her 
comments to yout editor. Send your corrections 
back to her with a single click. She can view just 
the latest corrections if she prefers. 


15 minutes 


Total: 21 minutes. 

Additional Benefits: Audit trail built 
and saved automatically. Changes can 
be exported locally for further testing. 
Project checklist maintained centrally 

and enforced during review. 


Save time every day with CodeReviewer 


P Quickly package code for review CodeReviewer automates the peer code review 
process so developers spend time looking at source 
code instead of managing the process itself. 
CodeReviewer is the only peer code review tool with 
support for multiple remote reviewers, automated 
audit trails, SCM integration, and custom reviewer 
checklists. 


P Automate audit trails 
P Enforce checklists 
P Never miss a change again 


P Review code remotely 


CodeReviewer is supported on Windows NT, 2000, 
XP, and 2003, It works with Perforce, SourceSafe, 
Subversion, CVS, and Vault version control systems. 


P Integrates with version control system 


P No server to manage 


Web: http://code-reviewer.com 
E-Mail: info@code-reviewer.com 
Phone: 877.501.5651 


@ SmartBear Software 


Copyright © 2004 Smart Bear; LLC. All Rights Reserved 
“SourceSafe” and “Windows” are trademarks of Microsoft Co. Perforce” isa trademark of Perforce Software, Inc. “Vault” is a trademark of SourceGear LLC. 


The economics are obvious: If two developers would 
each normally spend an average of 30 minutes on a code 
review, once a day, that’s 20 hours per month. At a fully- 
loaded developer cost of $150/hr, that costs you 
$3,000/mo. Code Reviewer cuts the time in half, which 


means you save $1,500 every month. 


Code Reviewer costs $499/developer—one time!—so 


you make your money back with the first month! After that 
it’s gravy... all the way to the bank. 


1 Ah, the days before SaaS, when all software was “lifetime,” 
though you’d attempt “maintenance and upgrades” sales in fu- 
ture years at 20% of the original price. Life is better now. 


You can’t afford not to buy it, right? 
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But this argument never worked, not once, even 
though the reasoning is sound and customers requested 
it. The reason is that these “savings” aren’t how budgets 


actually work. 


In a perfect world, if the software development organiza- 
tion really did produce more, higher-quality code, that’s 
undeniably valuable. But it’s also essentially impossible 
to measure, and is in fact not measured. So the “savings” 


are invisible, even if real. 


On top of that, budgets are siloed; the “Salaries” budget 
is separate from the “Tools” budget. Never once in my 
seven years selling software at Smart Bear, whether 10 
seats or 2500 seats, did a company make a decision in 
which savings in the “Salaries” budget resulted in an in- 
crease to the “Tools” budget. 


With Code Reviewer, the customer buys because they 
genuinely believe it makes their team more productive, 
even though they cannot in quantify ROI. Spreadsheets 


are worthless—they either believe, or they don’t. 
As the startup founder, what do you do about this? 


First, create so much value” that there’s no need to 
“compute” it. If so-called pay-back period really is less 


than a month, it’s obvious. Change someone’s workflow 


so drastically for the better that they can’t live without it 
whether it’s saving money or not. Improve your cus- 
tomers’ marketing campaigns so obviously and drastically 
that they don’t need a spreadsheet to understand its 


value. 


2 more leads, more sales, faster code, efficiency, time-savings, 


cost-savings, delight, corporate brand, personal image, societal 
impact 


Second, price according to ability and willingness to 


pay and the structure of your business model, rather 


than as a direct function of so-called “value.” The soft- 
ware development department has a budget for tools, 
and different companies have different ways of arguing 
internally for expanding that budget—you have to match 
those constraints. The marketing department might be 


willing to pay for services but not for tools. 


In any case, rarely is a simple “cost savings” equation the 


way you're going to win sales or set prices. 
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Startup identity & the sadness of a successful exit 


by Jason Cohen on February 26, 2013 


Many founders experience a profound and prolonged 
sadness after selling their company. But “not selling” might 
be worse. Maybe my story will help you. 


= 


ex = 
4 WW 
ANDERSON 


“No, it’s fine, I’ve just never seen 
coupons used in an acquisition.” 


My fingers trembled as I fed page 34 of 72 into the fax 
machine, deftly pressing the head of each page into its 
creaky jaws one at a time so that this shitty cheap ma- 
chine wouldn’t snag two pages at once, slantways, ob- 
scuring the precious scribblings adorning the footer of 


” 


each page where it read: “Seller’s Initials: 


This is what the last six years were all for. All the labor. 
All the risk. The brave face for the troops. The self-inflict- 
ed unflagging optimism despite little supporting evi- 
dence. All those sleepless nights worried about making 
payroll with a one-time revenue business model where 
every month you have to find new revenue from new 
customers. The 10,000 deliberate hours of becoming an 
expert in something. The experience you get just after 
you need it. The hard lessons you occasionally glean 
from experience. The inner doubt suppressed for the 
morale of the team. No salary followed by low salary. The 
“eat what we kill” mentality. The scrounging and scrab- 
bling and begging and fighting the assholes for those 


morsels of revenue, those crumbs of validation. 


It’s over. We did it. I did it. American dream? Check. 


The 73rd page spat out confirming the successful transfer 
of the previous 72. 


And then... sadness. 


A profound sadness. Not depression—not hopeless or 
rudderless—but a pure sadness, when your lungs sink 
into your belly, the punch-in-the-stomach of discovering 
your dog was hit by a car or that your dad is terminally 
ill. 


“What the fuck?” I thought, “Why am I feeling this? ’'m 
supposed to be feel... happy? I guess? Something other 
than this.” 


Almost all startup founders experience a deep and 
prolonged sadness after selling their company, even 


when the sale is an outrageous success. Why? 


The answer is important and fundamental for all startup 
founders, whether or not they intend to sell their compa- 


ny some day. 


“I sell my artwork on Etsy. Want to see?"—Barista at an 


Austin coffee shop 


If you ask her, “Who are you?” She would answer: “I’m 
an artist.” 


If you ask her, “What do you do for a living?” She would 
answer: “I’m a barista, but that’s just my day job. Want to 


see my artwork?” 


Is she a barista because she pours coffee for money? Is 
she a driver because she drives a car to work? Is she a 


maid because she cleans her own apartment? 
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No, she’s an artist because that’s what she really is. 
“Barista” is one of many necessary means to the ends, 
where the “ends” are the basic human needs, followed by 
creating art, I guess with some Maslow layers in- 
between. 


A startup founder lacks this distinction between per- 
sonal identity and work identity, and this is the key to 


the “sale-blues” phenomenon and other behavior. 


A startup is the founder’s personal identity. Startups are 
not something you do to make ends meet or a “necessary 


evil” en route to what you “really” want to do. 


A startup is an obsession. You do it because you couldn’t 
stop yourself. Because when you were doing anything 
else, you were thinking about it. That is the mark of 
“who you are.” Interviewers ask me “Why did you decide 
to do a second and eventually a fourth startup?” And the 
answer is “For the same reason that I started the first one 


—because it’s in my DNA and I have to do it.” 


What do you do in your spare time when you have a 
startup? What spare time? This is all your time. It’s not 
just the last thing you think about before falling a sleep, 
it’s the thing that won’t let you sleep. It’s first thing that 


trickles into your brain in the morning like The Matrix 


patterns filling the void. 


It’s why you can weather the painful thoughts like the 
ones whistling through my ears while I fed legalese into a 
rickety fax machine. You are consumed, this is your life, 


this is you. There’s no room for anything else. 


When you sell your company, others are quick to throw 
jabs like “So, you sold your baby?” Which means: “You're 


a sell-out.” (Or: “I’m jealous.” Or both.) 


It’s less like selling your baby and more like selling your 
own identity, now left with finding a new one. If there is 
a new one, and there’s no particular reason to think there 


will be. Which is part of the fear. 


Speaking of babies, this all sounds a lot like “baby 
blues”—depression caused by elevated levels of the en- 
zyme monoamine oxidase A—that 70% of women experi- 
ence after giving birth. A third of those women will expe- 


rience this for up to a year (postpartum depression). 


It’s characterized as a feeling of loss and of mourning, 
which is seemingly at odds with the arrival of a new life 
which is just the opposite—gain and celebration. This in- 
tellectual dissonance creates secondary emotional effects, 
specifically the devastating belief that “I must be a bad 


mom for being sad that I have a new baby.” 


Are the baby blues the same emotional effect as selling a 
company? Maybe not—I already don’t like saying “a 
startup is like a baby.” Besides, postpartum depression is 
triggered by the arrival of responsibility and, if you insist 
on the baby/startup analogy, selling your company is the 


departure of responsibility. 


But one thing that definitely is the same is that “feeling 
of loss and mourning.” A piece of yourself has been 


eviscerated, irrevocably. 


This is not only a normal feeling, it’s by far the majority 
School, out of 22 entrepreneurs who sold their compa- 
nies, every one of them experienced this effect. Some refo- 
cussed the energy into the Next Thing (almost always 
new ventures), but most took years to find that thing 
which could replace not only the excitement but identity, 
and many still haven’t found that Next Thing at all. Many 
wished they could have their companies back; a few did 
buy them back. 
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Selling their companies forced them to answer a difficult 
question: If you could do anything, what would you do? 
What’s really important to you, as opposed to a job? 


What do you really want to be when you really grow up? 


The problem is, the startup already was that thing! It’s 
a grinding, batshit-crazy, risky, irrational, epic adventure. 
You wouldn’t have done all that in the first place if that 
wasn’t already the thing you wanted to do. A startup is 


never the easiest career path. 


Does this lead to the conclusion that selling a startup is 


the wrong choice much of the time? 
No. And this is perhaps an ever sadder realization. 


It’s an easier decision if you’re saying “I can’t wait to start 
the next thing” but perhaps you’re thinking “I don’t know 
what Ill do with myself.” You must resist the urge to be- 
lieve that getting millions of dollars will make you ful- 
filled, or happy. “Money doesn’t make you happy” is 
cliché because it’s true. You need to understand how to 
be fulfilled regardless. 


But all that doesn’t mean selling is wrong. 


Just like a bad relationship that has to be ended even 
though it will be painful, especially if you genuinely love 
the other person: Just because it’s painful doesn’t mean 


you don’t need to do it. 


Building a company in year-one is completely different 
than building that company through year-five or -ten. 
The CEO’s job description changes over time, and so does 
the company, whether you sell it or not. Are you emo- 


tionally prepared for this as well? 


Are you OK with innovation taking a back seat to devel- 
oping scalable, mature processes? Are you OK releasing 
control in day-to-day operations to managers, and then 
releasing control of the managers to your executive 
team? Are you OK wresting yourself out of Visual Studio, 
out of Adwords, off the website, off the live chat, out of 
the sales calls, trusting your managers and not being that 


sort of meddling micro-manager boss that you yourself 


hate? Are you OK shouldering the burden of the liveli- 
hoods of dozens or thousands of families rather than 
“pulling 90 hour weeks” to push some code out the door 


with a co-founder, which now seems easy in comparison? 


The fact is, successful startups grow up. They grow into 
businesses and mature into sustainability, risk-avoidance, 
HR law, strategic planning, executive meetings, and all 
that. The founder-CEO is still steering the ship but it’s a 
different sort of ship. 


What to do? I wish the answer weren’t “it depends” or 
“every choice is a path of pain,” but often that’s what it 


is. 


In my case, it changed over time. At Smart Bear I didn’t 
want to lead a huge company. I didn’t want to relinquish 
Eclipse and my ability to check in code. I didn’t want to 
manage managers or figure out what changes, strategies, 
hirings, products, marketing, and sales were needed to 
make $100M/year. So for me I sold at a good point: be- 
fore I needed a C_O, but after the company was big 
enough to garner enough money to cross the Freedom 
Line. 


Now at WP Engine, I have new ambitions and inclina- 
tions. I am now that CEO! who manages managers, who 
sets vision and direction but not day to day operations, 
who worries about company culture but who doesn’t 
have SSH access to all the servers, and who is driving to- 
wards a company with products and a market and a team 


which we believe can indeed generate $100M/year~ 


1 Editor’s note: Written in early 2013, later that year the wonder- 
ful Heather Brunner joined as our CEO, still leading the compa- 
ny 10 years later, as what I’ve since called a “late-joining 
founder.“ 


2 Editor’s note: It did, at hyper-growth velocity. Now, in 2023, 
our revenue is many times that figure. 


That’s exciting to me. This is my new challenge. I will al- 
ways love writing code and getting a company from $0 to 
$1M/year. 
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But, today, right now, for reasons unknown even to me, 


this is who I am. 


I went to find the pot of gold 

That’s waiting where the rainbow ends. 

I searched and searched and searched and searched 
And searched and searched, and then... 

There it was, deep in the grass, 

Under an old and twisty bough. 

It’s mine, it’s mine, it’s mine at last... 

What do I search for now? 

—Shel Silverstein, Where the Sidewalk Ends 


Editor’s note: I would later develop a personal framework 


for avoiding burn-out and staying with the company for the 
long haul. That story and system might help you if you're 


struggling with this now! 
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More money if you do, more money if you don’t 


by Jason Cohen on January 7, 2014 


A business always takes more money than you expect, even 
when you take this fact into account. Here’s why. 


“Our investments in growth platforms allowed us to 
sustain momentum in numerous key metrics and 
leverage that into continued strategic collaborations. 
Then Larry here messed up big time!” 


“Always raise more money than you think you need.” 


—people asking you to take their money 


Actually, they’re right. It’s not a ruse devised by greedy 


vulture capitalists vying for extra points of equity. 
You do need more money than you think. Why? 


Today you have a business plan, featuring a plausible 
growth trajectory (neither too conservative nor too opti- 
mistic), and an associated cost structure to drive signups 
and service customers. It costs money to acquire a cus- 
tomer, but takes time for the monthly revenue to cover 
those costs, therefore costs rise faster than revenue, so 
you need a cash investment to fund that growth. Your 
projected cash low-point is $X, thus you need to raise $Y, 
not just to service the operational cash requirements, but 
to de-risk the company with additional investments you 
couldn’t otherwise afford to make. 


But what if, nine months from now, revenue isn’t growing 
quite as quickly as planned? You still signed that 5-year 
office lease, hired support, sales, and engineering, and 


brought in training, a comptroller, and a business opera- 


tions person. Marketing is over-spending to try to kick- 
start growth. You’re burning more money than you ex- 
pected, but the correct course of action is to forge ahead 
and let revenue catch up, not fire half your team and re- 


treat. So, you’ll need more money than you planned. 


But what if, nine months from now, revenue is growing 
more quickly than planned? You scramble to hire ahead 
in support, sales, and the additional demands on engi- 
neering. You trade money for speed—recruiting, contrac- 
tors, office space, finance. You incur customer on-board- 
ing costs faster than planned, so although in the long-run 
your revenue will more than make up for it, right now 
these unexpected new customers cost much more than 
their revenue contributions, and cash reserves plummet. 


So, you'll need more money than you planned. 


The only way you need the actual amount of money you 
thought you needed, is if everything in the next 18 
months goes exactly according to plan—a deviation in ei- 


ther direction means you need more money. 


You’re delusional if you think you can predict your start- 
up’s finances 18 months out with precision! Which is to- 


tally OK—you’re not supposed to. 


But not raising “more than enough” money means you’re 
denying that reality, and therefore causing a time-pres- 
sured cash-crunch for yourself 9-12 months from now. 


That’s irresponsible. 


As Douglas Hofstadter says: 
Hofstadter’s Law: It always takes longer than you ex- 
pect, even when you take into account Hofstadter’s 


Law. 


Similarly: 
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A business always takes more money than you expect, even when you take this fact into account. 
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You can have two Big Things, but not three 


by Jason Cohen on January 7, 2014 


No you can’t “have it all.” You can have two things, but not 
three. 


“Unfortunately, not only have they stolen your identity, 
they’re also living your best life.” 


Forget work/life integration for a minute. How much 


time do you have, regardless of partitioning? 


From your 24-hour daily allotment, the 1950s-style 
break-down is 8 hours for work, 8 for home and com- 
mute, and 8 for sleep and ablutions. So, “work” and 
“home” are the two things in which you can spend 40+ 


hours per week. 


This is the amount of time it takes to tackle something 


huge. A career. A parent. A startup. 


There are weekends and vacations and sick days and 
such, but those don’t add up to enough concentrated time 
to carry off something like a startup without causing 


work or home to suffer. 


6. 


Of course “work” and “home” are just placeholders for 
“Big Things.” If you’re unattached, “home” doesn’t occu- 


py significant time. 


The rule of life is: You can have two “Big Things” in 


your life, but not three. 


Big Things include: 


e Job 

e Kids 

e Spouse 

¢ Social Life 

e Major Hobby (e.g. build a boat in the garage, become 
a chess master, video game addiction) 

e Startup 


You can do a startup on the side while you have a day 
job, but your family will never see you. You might even 
lose your family. It happens. This is partly why it’s easier 
to start a company before you have a family or even a 


spouse. 


You can have a job and a social life, but unless your 
spouse is fully integrated and agreeable to that social life, 
there will be strife. “Going out with the guys again?” 


Yes, “kids” and “spouse” are on the list separately. Young 
kids strain marriages because there’s not enough time to 
invest in the kids as well as be there for each other. 


“Someday we're going to look back on false memories 
of this and laugh!” 


Some people try to “have it all.” Men and women both. 
But it’s never true. At most two can function well; the 


rest do not. More often, there’s just one that receives the 
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majority of the energy, and the rest suffers. 


Note that “Sleep” isn’t on the list of options, even though 
it’s mathematically the same in terms of time commit- 
ment. That’s because cutting out sleep doesn’t work— 


then you can’t function at a high level at anything. 


No, you are not an exception. That’s egotistical self-de- 
ception. Not on sleep, and not on the number of Big 


Things. Ask the people around you if they think you’re 


not failing at one of your Big Things. 


Time to decide which two. 
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by Jason Cohen on January 28, 2014 


Software cannot scale through architecture and automation 
alone. New, more difficult problems appear that didn’t exist 
before, also bringing new downstream consequences. 


“The idea is to not only grow our piece of the pie, 
but to expand into other nearby pastries.” 


Something interesting happens when you run more than 
1,000 servers, as we do at WP Engine}, powering hun- 


dreds of thousands of websites. 


1 Editor’s Note: In 2023, this article is nine years old, we run 
more than ten times as many servers, and the lesson of this ar- 
ticle continues to be accurate. 


Suppose I told you that on average our servers experi- 
ence one fatal failure every three years. The kernel pan- 
ics (the Linux equivalent of the Blue Screen of Death), or 
both the main and redundant power supply fails, or some 
other rare event that causes outage. Does that sound like 


poor quality? 


Think about the laptops and desktops you’ve owned. 
Some last longer than others, but it’s probably something 
like 2-4 years before the OS locks up or a battery can’t 
hold a charge anymore. Now consider your laptop is idle 
the vast majority of the time, whereas our servers are 


getting pounded multiple times per second, 24/7. Even 


in the wee hours of American timezones when our pan- 
company traffic dips to its nadir, we’re running malware 


scanners, off-site backups, and other maintenance. 


So, even if our servers are more hardy than your 
MacBook Pro, they're taking 100x the beating, so one 
failure every three years seems pretty reasonable. 


Windows NT crashed. 

I am the Blue Screen of Death. 

No one hears your screams.” 
—Haiku from FSF 


But remember, we have 1,000 servers. Three years is 
about 1,000 days. So that means, on average, every sin- 


gle day we have a fatal server error. 


Not to mention 10 minor incidents with degraded perfor- 
mance, or a DDoS attack somewhere in the data center 
affecting our network traffic, or some other thing that 
sets pagers a-buzzing in our DevOps team and mobilizes 
our Customer Support team to notify and help 
customers. 


“Well sure,” you say, “that’s normal as you grow. If you 
had just 10 servers and 100 customers, you’d have much 
fewer problems and many fewer employees. Today you 
have more customers, more servers, and more employ- 


ees. What’s so hard about that?” 


The insight is that that scale causes rare events to be- 
come common. Things happen with 2000 servers that 
you never once saw with 50 servers, and things which 
used to happen once in a blue moon, where a shrug and 
a manual reboot every six months was in fact an appro- 
priate “process,” now happen every week, or even every 


day. 
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Things as rare as, well, you know... 


A 


ae i 
= _—O me J 
Ge “i ls 


“Boy, what a day, huh? 


It’s not only problems that morph with scale, but your 


ability to handle problems. 


For example, a dozen minor and major events every day 
means 20-50 customers affected every day. Now consid- 
er what happens as we try to inform 50 customers. For 
some we won't have current email addresses, so they 
don’t get notified. Some of those will notice the problem 
and create extra customer support load at minimum, but 
at worst theyll post on Twitter about how their website 
was slow or offline today and WP Engine “didn’t even 
know it.” Then our social media team has to piece all this 
together, attempt to respond, maybe put together a spe- 
cial phone call with that customer, and so on. Those cus- 
tomers are also more likely to leave a bad review on 
some review site, compared with the 99.99% of cus- 
tomers who experience no such incident, but also had no 
reason to decide that “today is the day I leave a good 


review.” 


Or consider the scale-ramifications of on-boarding 1,000 
new customers a month. In that case, it’s likely that any 
given server issue will affect a customer who has only 


been with us for a month or two. Thus the issue causes a 


“bad first impression,” which is harder to address than a 
customer who has been with us for three years and there- 


fore has built up a bank account of patience. 


So, rare things being common isn’t just difficult from the 
operational side, but also when you try to handle those 
problems with customers or other downstream conse- 
quences, causing much more work to solve than when 


the company was small. 
The usual response to this is “automate everything.” 


As with most knee-jerk responses, there’s truth in it, but 
it’s not the whole story. 


Sure, without automated monitoring we’d be blind, and 
without automated problem-solving we’d be over- 


whelmed. So yes, “automate everything.” 


But some things you can’t automate. You can’t “auto- 
mate” a knowledgable, friendly customer support team. 
You can’t “automate” responding to a complaint on social 
media. You can’t “automate” the recruiting, training, rap- 
port, culture, and downright caring of teams of human 
beings who are awake 24/7/365, with skills ranging 
from multi-tasking on support chat to communicating 
clearly and professionally over the phone to logging into 
servers and identifying and fixing issues as fast as (hu- 


manly?) possible. 


And you can’t “automate” away the rare things, even the 
technical ones. By their nature they’re difficult to define, 
hence difficult to monitor, and difficult to repair without 


the forensic skills of a human engineer. 


Does this mean all our customers have a worse experi- 
ence? No, just the opposite. Any one customer of ours has 
fewer problems per year now than a year ago, because 
we're constantly improving our processes, automation, 
hardware, and human service. It’s when you look across 
the entire company, and the non-linear additional effort 
it takes to not just improve the average experience, but 
to manage the worst-case experience, that you appreciate 
the difficulties. 
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Does that give high-scale companies like WP Engine an 
excuse to have problems? No way! In fact, if we’re not 
constantly improving on all fronts, the scale itself will 
catch up and overtake us, so we have to adhere to the 
laws of automation and diligence even more than small- 


er, slow-growing ones. 


But for those of you in the earlier stages of your compa- 
nies, when you project 5x growth coupled to just 5x the 
costs (or only 3x the costs because you'll get cost-savings 
at scale), you’re guessing low. When you show 5x growth 
in projections but don’t budget for new hires in areas like 


security, technical automation, specialized customer ser- 


vice areas, and managers and executives who have trod 
this path before and come battle-hardened with play- 
books on how to tackle all this, you’re heading for an 


ugly surprise. 


And with high growth, the surprise appears quickly, and 
recovery means acting twice as fast again to claw back 
ahead of the effect. 
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The only way to guarantee startup success 


by Jason Cohen on July 29, 2014 


What is it like to reach the pinnacle of success? Is that 
where you attain happiness and fulfillment? Or are those 
found right here, right now. 


“As long as it has all its fins and is a remorseless 
eating machine, we'll be happy.” 


Everyone said this would be the most embarrassing mo- 


ment in the band’s eight-year career. 


Depeche Mode had decided to play the Pasadena Rose 
Bowl—capacity 60,000—for the 101st show of their 
1988 tour. To sell out would make it one of the largest 
music concerts ever played in America—highly unlikely 
for an English electronic band. Claiming they were popu- 
lar enough to fill that stadium was an audacious act of 
bravado that critics were eager to see transform to humil- 
iation as they played to a vast, near-vacant space, 
mocked by the each of the tens of thousands of empty 
seats. 


KROQ DJ Richard Blade knew that secretly the band 
themselves were doubtful. As he sat with singer/writer 
Martin Gore in the empty stadium before tickets went on 
sale, Martin “confided in me that he was nervous and 
hoped they could at least sell out the floor seats—just 
10,000 tickets.” 


But the show sold out, with paid attendance greater than 
any Rose Bowl event in the eight preceding years. It was 
the defining moment of the band’s career—the moment 


when they undeniably “made it.” 
And yet, simultaneously, one of the saddest moments. 


Again from Blade: 


“Backstage, after their amazing performance, I chatted 
with [lead singer] Dave Gahan as he cried from pure 
happiness. He told me that the tears were because he 
didn’t know if the group could ever pull off anything 
this great again and for him it was the most emotional 
concert of his career.” 


Indeed, they never would pull off anything that great 
again, even though they increased record sales, wrote 
more hits, released more albums, played more tours, 
even reforming the band after the loss of one member 
and through multi-year battles with drugs, alcoholism 


and depression from the three remaining members. 


It’s true though, what could top that moment? After 
you’ve proved everything that could be proved, to the 


critics, to your fans, and even to yourself? 


What does it mean for you or me, that reaching the 
pinnacle of success is not only strikingly fleeting, but also 
unhappy? That tears of joy transmute immediately to 
tears of sadness, because reaching the peak means by 


definition your next steps must be downhill? 


The same thing happens at startups. It’s well-documented 
that immediately following the “success” of an exit, 
founders almost inevitably fall into a sadness and even 
depression, as I’ve written about before. 


It could mean this is all for nothing. That our fleeting 
moment of noon-time glory is cruelly bookended by years 


of gut-twisting emotional mountain-climbing in the a.m. 
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and a meandering, permanently unfinished quest for 


meaning and purpose in the p.m. 


It does mean this, if you let it. The way you let this hap- 
pen is to believe that the goal is to achieve a single 
moment of success—a big sale, going public, or passing 
it on to your daughter. 


Rather, you must understand that it is the building, not 


the result of that building, that matters. 
Let’s break it down: 


You spend 99.99% of your time on the journey of build- 
ing a startup, and 0.01% basking in the temporary eu- 
phoria of “success,” such as selling it to someone else or 
ringing the bell on the morning of your IPO. 


If that moment even comes, which it most likely will not. 
Often 100% of your time is spent on the journey, 0% in 
euphoric ecstasy. 


If you look back and say “That was a wonderful time in 
my life. ’'m glad I did it, and I'd do it again. Sure I would 
do some things differently—is that not true of anything? 
—but I’m proud of what I did and I’m stronger and wiser 
today than ever before. I created terrific jobs, where 
great people were empowered to build important things 
together, who were comfortable and safe, who shared in 
the bounty of whatever upside we could muster, and who 
themselves would also say they’re personally fulfilled just 


as I am.” 
Then you win. 


But if your measure of “success” is based on a specific 
outcome—based on metrics, or money, or growth-rate, or 
number of employees, or whether you had an exit, or 


how much money you raised, or how many humans you 


cajoled into sticking their nose in your app, then mathe- 
matically you’re almost guaranteed to fail, but only be- 
cause of your own definition of failure. You’re guaranteed 
to spend years of your life in nervous agony, chasing an 
outcome you think will make you happy instead of mak- 


ing all those years be the years you are happy. 
Then you lose. 


Don’t get me wrong—I love making money, I’ve made 
plenty from previous startups and I hope to make plenty 
more at WP Engine, and so do our employees (all of 
whom are shareholders) and our investors. And honoring 
metrics is part of building a high performing team and 
building a huge, sustainable enterprise. Nothing wrong 


with using metrics and money to keep score in this game. 


Keep score, so long as you can distinguish between the 
game and life. Keep score, while also basking in the thrill 
of generating happy customers and launching unique 
products and gathering the energy and brainpower of 


brilliant humans tackling interesting problems. 


In fact, ’'d argue that focusing on the nature of the jour- 
ney solves one of the great riddles facing all startups. To 
paraphrase Peter Thiel: the first employee joins because 
it’s the ground floor of an exciting startup, but why will 
the 20th person join? Or the 200th? 


To be part of the journey. 

That’s why you should be there too. 

If you value the journey, your “success” is guaranteed. 
Printed from: A Smart Bear 
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In its emptiness, there is the function of a startup 


by Jason Cohen on August 12, 2014 


Everything about a startup changes over time. The few 
things that don’t, are its essence. The voyage is 
meaningless, unless you decide what those things are. 


“I didn’t say I was unfulfilled. I said I was hollow.” 


Calm your frenetic TL;DR twitchy-Twitter-brain for five 
minutes, and let some Chinese philosophy wash over 


you. 


Thirty spokes join in one hub 
In its emptiness, there is the function of a vehicle 


Mix clay to create a container 
In its emptiness, there is the function of a container 


Cut open doors and windows to create a room 
In its emptiness, there is the function of a room 


Therefore, that which exists is used to create benefit 
That which is empty is used to create functionality 


—Lao Tzu, Chapter 11, Tao Te Ching, ~600 BCE, trans- 
lation by Derek Lin 


An ancient brainteaser asks: If a wooden ship sails 
around the world, but along the way each board rots and 
is replaced by new boards, so that by the end of its jour- 
ney every board has been replaced, is it still the same 


ship? Did “it” really go around the world? 


Lao Tzu suggests the answer is “yes,” because the func- 
tionality of the ship—the “emptiness” created by those 
boards—is the essence of the ship. 


What about yourself? Your boards are being replaced 
every second—trillions of cells splitting, growing, dying. 
Are you the same person you were ten years ago? 
Physically, not really. In DNA yes (mostly), but identical 
twins share DNA and are not the same person. Can you 
articulate precisely what resides in the emptiness created 
by your body, or by the electro-chemical pandemonium 


sloshing within your skull? 


What about a startup? It’s not a person but composed of 


people. Is a company of 2 people and 150 customers the 
“same company” as with 150 people and 25,000 cus- 
tomers? In tangible ways it’s not the same, but in its 
emptiness lies the essence and purpose of this motley 


crew of human beings. 


So the interesting question for a startup is: What is con- 
stant? What doesn’t change as we grow, and thus what 
binds us together in this journey? If we “go around the 
world,” changing out every board, what has been 


conveyed? 


Can’t be about product features, they change, grow, and 
die like cells. Can’t be about marketing prose, that’s 
molded by A/B testing into market-resonance. Can’t be 
people—the team’s composition changes, and those who 
stay for the entire voyage themselves change. Can’t be 
the metrics, they'll change by orders of magnitude. Can’t 
be the logo, or even the name. Can’t be the office space— 


Lao Tzu already made that clear, as did COVID. 


So, what doesn’t change? Whatever it is, that is what de- 
fines the startup and it’s purpose, and is the answer to 
the all-important question: Why should anyone—employ- 
ees and customers alike—join the founder on this jour- 


ney, sharing this very personal responsibility? 


At WP Engine we call these constants our “values”—our 
inviolable constitution. When we need to make a tough 
decision, our values determine the outcome. For exam- 
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ple, if in some choice we must either lose money or do 
what’s right for the customer, we will do what’s right for 
the customer. Or if we have to decide whether to fire 
someone who is exceptionally skillful at his job but is dis- 
honest or works for himself instead of for the team, we 


will always fire that person. 


In fact, a clear agreement of core values is how you em- 
power everyone in the company to act like an owner. 
Can someone in their first week of tech support decide 
whether it’s OK to give a customer a large refund or 
spend extra hours to go above and beyond to fix some- 
thing? If you have a framework of values which clearly 
points to that as the right course of action, then someone 
can make that decision without approval-chains and 
meetings and debates. Isn’t that the definition of empow- 
erment? And doesn’t that sound like part of the solution 


to “scale?” 


Signing the WP Engine values 


This is why it’s so important to encode your compa- 


ny’s values on paper. For us, we literally put them on 


the wall and we all signed it like the US Declaration of 


Independence. You have to ensure everyone not only 
agrees to them as a condition of employment, commit- 
ting them to memory due to external requirement, but 
because in their bones they already embody those values, 
and therefore will naturally continue propagating them 


to those around them and those who follow. 


“Company Culture” is the output of living your val- 
ues. It’s never perfect, either as individuals or an organi- 
zation, and at any given time you're falling short at some 
aspects of the culture, aspirationally trying to live up to 
your own expectations. That’s healthy. 


The three companies I founded before WP Engine were 
successful enough, but I never thought about culture or 
values. That was a mistake. Today two of those compa- 
nies still exist, and after nearly two decades have contin- 
ued to grow, and yet, without a higher purpose, a truly 
important mission for our customers and dedication to 


our employees, maybe who cares? 


Why so dismissive of that success? Because what sort of 
jobs did that create? Were people happy and empow- 
ered? Mentoring and being mentored? Did they build a 
career theyre proud of? Did they feel that this time in 


their life was not wasted? 


I don’t actually know, how sad is that? Financial success 
is the means, not the ends. Without financial success 
the company isn’t sustainable, so that component of suc- 
cess is absolutely necessary. But “money doesn’t make 
you happy,” and it certainly doesn’t make other people 


happy, or lead to a fulfilling, meaningful life. 


What does? Answer that question, revealed by your cul- 
ture in the emptiness created by your values, and you’ve 
truly done something important with your life, and for 


others’ lives. 


What is your emptiness? 
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by Jason Cohen on November 5, 2014 
Pricing is inextricably linked to brand, product, and 


purchasing decisions. It cannot be “figured out later,” because 
determines your business model today. 


|| Histens) Helc, s 
Clothing||_ 


C _Stasn 


“We're not changing anything, 
we just wanted to charge more.” 


It’s often said that you shouldn’t talk about price during 
customer development interviews. The usual justification is 
that your goal is to uncover the details of your potential 
customers’ lives and pain-points, whereas mentioning price 
diverts the discussion to budgets and operational things far 


removed from the customers’ day-to-day life. 


But I disagree. Price is as important as any other feature to 


determine product/market “fit.” 


How many times have you heard someone agree that “it 
would be great if someone did X,” but when you show 
them a demo of X, but it costs $700, and they don’t buy? 
Or seen a review of an iPhone app hung up on pricing trivi- 
alities: “It would be pretty good at $0.99, but it’s not worth 
$1.99.” How many times have you seen someone struggle 
with an inferior product because they cannot afford the 


better one? Or struggle with the freemium version because 


they refuse to pay anything at all, even though they like 
and use the product? Or struggle with an inferior, expen- 
sive product that was purchased based on the salesmanship 
—“expensive must mean _ it’s better’—instead of 


craftsmanship? 


Price is inextricably linked to brand, product, and pur- 
chasing decisions—by whom, why, how, and when. Price 
is not an exercise in maximizing some micro-economic sup- 
ply/demand curve, slapped post facto onto the product. 
Rather, it fundamentally determines the nature of the prod- 


uct and the structure of the business that produces it. 


Consider the consequences of each order-of-magnitude of 


pricing: 


$0/mo means your goal is to maximize growth (trust and 
usage) instead of revenue. Your product is designed with 
natural tripwires to trigger other pricing (Freemium _mod- 
el), or not (business model left as an exercise to your fu- 
ture self). Typically requires venture funding because you 
have no income, and if you’re successful you'll need lots of 
people and tech to run the business. Even super-efficient 
outliers like WhatsApp (sold for $19B with 55 employees) 
and Instagram (sold for $1B with 13 employees) each 
raised tens of millions of dollars in venture funding. This is 
often B2C because the value is in quantity of customers, 


and there are 100x more consumers than businesses. 


$1/mo means you can’t afford customer service and it 
must incrementally free to run the technology behind it, 
both of which have implications for the sort of product you 
have to build (e.g. simple enough to be self-service). 
Marketing and sales spend is nil (at least initially), so there 
has to be a reason it spreads by word of mouth, ideally vi- 
rally as a natural result of using the product itself, or 
through vocally passionate early users, perhaps galvanized 


around a mission. 
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"And for the connoisseur on a budget we have a 
lovely Tuesday vintage at a buck fifty." 


$10/mo means people see you as a cheap version of some- 
thing else, but still expect a phone number. (Think: 
GoDaddy). Bootstrapped businesses can make this work 
(e.g. most shared hosting companies), but they only make 
interesting money at large scale (by definition, because it 
takes over 8000 customers to make only $1lm/yr in 
revenue), which takes a long time to grow. So you can get 
to $10s of millions but it will take 10 years. (Again, like 
shared hosting companies.) This is a hard slog. If you want 
to scale faster you'll need venture funding, both because of 
the anemic revenue, and because otherwise you can’t af- 
ford to advertise. Often bootstrapped companies of this 
type boast about having no marketing or sales depart- 
ments, but the truth is they can’t afford it, and those com- 
panies typically grow slowly, often eclipsed by companies 
who can afford to grow 10x faster. In a huge market this is 
probably still OK because there’s enough customers for 
everyone to thrive in different ways. On the good side for 
this business model, often people will simply forget they 
have the $10/mo service even if they don’t use it, resulting 
in “free revenue” which at scale can be surprisingly sub- 


stantial (yet again like shared hosting companies). 


$100/mo means people expect to be able to call support, 
and if a competitor is substantially better, it’s probably 
worth the effort to switch. Also this is almost exclusively 
B2B unless it’s something “luxury.” Big companies can buy 
it without much consideration, but small companies need 
to understand the value, so you might need sales material 


to convince them, and a demo. I love this price zone for 


bootstrapped companies, because it’s low enough that you 
can address a broad business market, but high enough that 
you can get nicely profitable with a reasonable, achievable 


number of customers (e.g. 200-300), and you can afford to 


$1,000/mo means only medium to large companies can af- 
ford it. They'll have buying processes around annual bud- 
gets, approvals, ROIs, demos. You will be compared to al- 
ternatives and weighed. You need to be a part of that con- 
versation which means a real sales force, sales materials, 
impressive logos, case studies, and referenceable cus- 
tomers. Features might need to include things that big com- 
panies need that others don’t, like role-based access privi- 
leges and integration with LDAP This can be a surprisingly 
difficult zone to become profitable in, because the sales 
and marketing motions and engineering costs are the same 
as for much larger sales, but without the attendant 


revenue. 


ANDEZSIN 
“T like it, but I’m looking for more of a 
status symbol. Any way you can double 
the price?” 


$10,000/mo means larger companies only. It’s unlikely the 
product is sufficient out-of-the-box, so you might need in- 
house professional services or to partner with consulting 
firms for implementation—constructing the “Whole 


Product” in the language of Geoffrey Moore. If something 


goes wrong they’ll cancel and not be willing to pay out the 


rest (no matter what your contract says; what are you go- 
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ing to do, sue your customers?). It’s possible to achieve this 
price-point with mid-sized companies if it’s usage-based 
(e.g. ZenDesk, JIRA, Box, Slack) or performance-based 
(e.g. marketing optimization), so the product itself needs 


to create and then demonstrate value to earn that result. 


$100,000/mo means Global 2000 only, large-scale projects 
that require multiple departments for decision and ap- 
proval, long sales cycles (9-18 months) which requires 
massive cash outlay as you bide your time. Likely starts as 
a pilot or proof-of-concept, which hopefully you can get 
paid for, and which needs to be convincingly successful, 
perhaps battling against another pilot with another vendor. 
You'll need on-site visits even in this day and age of video- 
conferencing. Examples: WorkDay (much of revenue is con- 
sulting), IBM. 


So which is better—higher or lower or in the middle? 
That’s a better question. Here’s something resembling an 


answer. 


The main thing is to understand that price is linked to 
everything, and bring it into your customer interview 
process. Talk about their expectations of cost, and why, 
who would write the check, who would need to approve it, 
whether an ROI argument would be welcomed or scoffed 
at, or what would need to be changed (if anything?) to jus- 


do that an in fact double your price). 


Price is not an afterthought, it is essential business design. 
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Your non-linear problem of 90% utilization 


by Jason Cohen on December 8, 2015 


Is everyone working very hard, all the time, and yet 
accomplishing 1/1 0th of what it seems they should? Maybe 
this is why. 


Wii 


ANDERSON 
“You've got a breakfast meeting, meetings all morning, 
two lunch meetings, your afternoon meetings, 

a dinner meeting, and then it looks like you've 
pencilled in ‘self-immolation’ after that?” 


Suppose a web server is running at 50% of its full capaci- 
ty. Browser traffic doesn’t arrive in regular, smooth 
amounts; it comes in spurts and occasionally large spikes. 
Because the server is under-utilized, when a spike arrives 
there are spare resources to handle the increase. If the 
spike is sufficiently large, performance will degrade, and 
if larger still, many of the requests will be rejected rather 
than answered; after all, there’s some limit past which the 


server cannot do any additional work. 


Now suppose someone looks at a report that shows “50% 
utilization” and says “Hey now, this is a server, not a per- 
son! It costs us the same whether we drive it at 50% uti- 
lization or 90% or 99%. So let’s get our money’s worth 


and drive it into the 90s!” 


What happens? Even normal variations in traffic will 
drive the server past its capacity. The average time to re- 
spond to a request will skyrocket, and often requests will 


be dropped altogether. Not due to an unusual event, but 


all the time. The system is now brittle—not good for 
costs, not good for the quality of the product or customer 


experience—just bad all around. 


Maybe we can drive high utilization by having multiple 
servers work as a team. Suppose we have three servers, 
all serving traffic for the same website, all at 70% capaci- 
ty. That sounds like a happy medium between 50% ca- 
pacity (wasting money) and 90% (brittle). The total 
amount of utilization is 2.1 servers (3 x 70%), so we’re 


nicely over-powered for traffic spikes. 


But what happens when one server runs into problems? 
Suppose it crashes, or the power in its data center cuts 
out, or someone else breaks the network with a glut of 
garbage traffic. The 2 remaining servers now have to 
deal with 2.1 servers’ worth of traffic. Both are at 105% 


capacity, and we’re back to broken and brittle. 


This isn’t really about servers; it’s about you and your 
teams. It’s about how your “busy” life not only diminish- 
es your productivity, but how your whole team is hectic, 


yet bringing itself to a crawl. 


We all have a capacity, whether you want to measure it in 
hours, in energy, in focussed attention, or if you don’t 
want to measure it at all. Instead of web-requests, we 
have life-requests, whether those are inbound emails, 
Jira tickets, Zendesk tickets, Salesforce leads, requests 
from co-workers, requests from a friend, or families that 
need our time and attention even more than they need 


our paycheck. 


90% utilization is causing more failure than you real- 
ize, not just in burn-out, but in productivity and out- 
put. Of course you'll burn yourself up, sacrificing sleep, 
health, friends, family, and other things you mistakenly 
take for granted, but I suppose you knew that already. 


Youre trading that for super-human productivity, right? 
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“I’m sorry, I was sure there was something over there 
that I needed to bark at. Please continue.” 


But you won't even receive outsized professional gains as 
a reward. This condition is a combination of frequent 
context-switching and  interruption—the Twin 
Enemies of productivity. Work-completion will drag out 
because it’s constantly interrupted. Some will be 


abandoned. 


Worse, in many organizations everyone is operating at 
90%, which then reacts like the three-server system, 
where the inevitable hiccup from any one person causes 
a ripple effect that hurts several other people or projects. 
Since they are over capacity, rather than absorb the spike, 
they too will ripple the problem to others—a cascade like 


the the run-away chain reaction of an atom bomb. 


The key word here is inevitable. People get sick or die or 
leave or change or have to run an errand or want to do 


even one minor piece of work that wasn’t mapped out 


weeks in advance. True emergencies arise that deserve to 
interrupt work. This is not something you can “architect 
out” of the universe; rather, you need to build a system 
that assumes variation and interruption, and design your 
personal and team’s work-style to be resilient to that 


variation. 


The ideal is probably a situation where most of the time 
you’re in the safe zone, with occasional surges into high 
gear for a short period of time and for good cause. For 
example, a brand new product launch is usually attended 
by some extra time fixing bugs, especially post-launch 
where it hits real customers and a few issues are discov- 
ered that we all agree should be fixed swiftly before 
more customers encounter it. Or there could be a clear- 
and-present danger to the company that requires a spe- 
cial, time-bounded rally. Or you could use infrequent and 
brief surges in a fun way, like a Hack-a-Thon or a Bug 


Squash Competition or a Ticket Kill Day. 


We're erring on the side of over-utilization, and rather 
than providing the benefits of competitive advantage 
through higher productivity, it’s creating needless turmoil 
and lower productivity. 


Don’t let yourself, or your team, fall into the trap. 
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How to measure the accuracy of forecasts 


by Jason Cohen on June 28, 2016 


How do you assess forecasts, when the forecast is only a 
probability? It’s not just about accuracy. Let’s dive into the 
math. 


RANDOM GUESS 
— ANDEZSon 


“And now the 7-day forecast...” 


“There’s a 30% chance of rain today.” 
And then it didn’t rain. So, was the forecast accurate? 
Or it did rain. Is the forecast inaccurate? 


How do you hold forecasters accountable, when the fore- 
cast is itself a probability? The answer appears tricky, but 
ends up being simple enough to answer with Google 
Spreadsheets. 


It’s a journey worth taking because of the value of build- 


ing better forecasts: 


¢ Lead scoring: Putting a value on a new sales lead, 
predicting the chance of converting to sale, and its 
dollar value. 

¢ Predicting churn: If you could predict the chance 
that a given customer will churn in the next thirty 
days, you could be proactive and perhaps avert the 
loss; do this enough and you're on the road to 


Product/Market Fit. 


¢ Predicting upgrades: If you could predict the chance 
that a given customer is amenable to an upgrade, 
you could focus your internal messaging efforts 
accordingly. 

e Risk assessments: Establishing better probabilities 


on risks results in more intelligent investments. 


So how do you measure the accuracy of a prediction that 
is expressed as a probability? Let’s return to the 


meteorologist. 


Accuracy Error 


Clearly, a single data point tells you nothing. 


Rather, the correct interpretation of “30% chance of rain” 
is the following: Gather all the days in which the meteo- 
rologist predicted 30%. If the meteorologist is accurate, it 
will have rained 30% of those times. Similarly, the fore- 
caster will sometimes predict 0% or 10% or 50%. So we 
should “bucket” the data for each of these predictions, 


and see what actually happened in each bucket. 


What is the right math to determine “how correct” the 
forecaster is? As is often the answer in statistics, we can 
take the squared difference ' between the forecast and the 
actual result. 


1 Why do we square the errors instead of using something sim- 
pler like the absolute value of the difference? There are two 
answers. One is that squaring the differences intentionally ex- 
aggerates items which are very different from each other. The 
other is that the mathematics of squared differences is much 
more tractable than using absolute value. Specifically, you can 
expand and factor squared differences, and you can use differ- 
ential calculus. Computing a linear regression line with least- 
squares, for example, is derived by using calculus to minimize 
the squared differences, but that same method cannot be ap- 
plied to linear differences. Nassim Taleb is famously against 
this practice, but let’s not argue the point now. 


Suppose we have two forecasters, and we ask: Who is 
most accurate? “Accuracy Error” is measured by the sum 


of the squared differences between the forecast and reali- 
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ty. Whoever has the least total error is the better 


forecaster. 


For example, suppose on some given set of days, forecast- 
er A always predicted a 32% chance of rain, and B al- 
ways predicted 25%, and suppose in reality it rained on 
30% of those days. Then the errors are: 


Predict Actual Squared Diff = Error 
A 32% 30% (0.32 — 0.30)? = 0.0004 
B 25% 30% (0.25 — 0.30)? = 0.0025 


It feels like we’re finished, but alas no. If all we compute 
is Accuracy Error, we fall for a trap in which we easily 
forecast with perfect accuracy, while also being utterly 


useless. 


Discernment 


Suppose these meteorologists are in a region that rains 
110 days out of every 365. That is, the overall climactic 
average probability of rain is 30%. A meteorologist 
would know that. So, a meteorologist could simply pre- 
dict “30% chance of rain” every single day, no matter 


what. Easy job! 


Our Accuracy Error metric will report that this forecaster 
is perfect—exactly zero over a whole year of predictions. 
Because, the prediction is always 30%, and indeed on 
those days it rained 30% # £of the time: 
(0.30 — 0.30)? = 0 . Except the forecaster isn’t perfect; 
she’s not forecasting at all! She’s just regurgitating the 
climactic average. 


ANDERSON 


“But, to be fair, there's a fifty-percent chance 
of just about anything.” 


And so we see that, although “accuracy error” does mea- 
sure something important, there’s another concept we 
need to measure: The idea that the forecaster is being 
discerning. That is, that the forecaster is proactively seg- 
menting the days, taking a strong stance about which 
days will rain and which will not. Staking a claim that 


isn’t just copying the overall average. 


There is a natural tension between accuracy and discern- 
ment which becomes apparent when you consider the 


following scenario: 


Suppose forecaster A always predicts the climactic aver- 
age; thus A has 0 accuracy error but also 0 discernment, 
and is therefore useless. Now consider forecaster B, who 
often predicts the climactic average, but now and then 
will predict 0% or 100% of rain, when he’s very sure. 
And suppose that when he predicts 0% the actual aver- 
age is 10%, and when he predicts 100% the actual aver- 
age is 90%. ie. when “B is very sure,” B is usually 


correct. 


B will have a worse accuracy error score, but should have 
a better discernment score. Furthermore, you would pre- 
fer to listen to forecaster B, even though he has more error 
than A. So the idea of “discernment” isn’t just a curiosity, 
it’s a fundamental component of how “good” a forecaster 


is. 
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How do you compute this “discernment?” We once again 
use squared differences, but this time we are comparing 
the difference between observed results and the climactic 
average, i.e. how much our prediction buckets differ from 
the climactic average, and thus how much we're “saying 


something specific.” 


For both accuracy error and discernment we use the 
weighted average for each prediction bucket. Let’s start 
by computing accuracy error for forecaster B, assuming 
that out of 100 guesses, 8 times he predicted 0%, 12 
times he predicted 100%, and the remaining 80 times he 


predicted the climactic average of 30%: 


Actual 
in this Accuracy 
Bucket bucket Error 


0% 10% (0.00 — 0.10)? = 0.01 
30% 30% (0.30 — 0.30)? = 0.00 
100% 90% (1.00 — 0.90)? = 0.01 


Now we weight the errors by the number of predictions 


in each bucket, yielding the final Error as the weighted 


average: 
Weighted 
Bucket Accuracy Error # in Bucket Error 
0% 0.01 8 0.01 x 8 = 0.08 
30% 0.00 80 0.00 x 80 = 0.00 
100% 0.01 12 0.01 x 12 =0.12 
Total 100 0.20 
Average 0.20/100 


= 0.002 


Forecaster B’s accuracy error of 0.002 is still low. Now 
we'll compute this new discernment score, which is 
exactly like accuracy error, except instead of the squared 
difference between “predicted” and “actual,” it’s the 
squared difference between “climactic average” and 


“actual”: 


Actual 

in this 
Bucket bucket Discernment 
0% 10% (0.30 — 0.10)? = 0.04 
30% 30% (0.30 — 0.30)? = 0.00 
100% 90% (0.30 — 0.90)? = 0.36 


And then creating the weighted average, just as with 
accuracy error: 
Weighted 


Bucket Discern. # in Bucket Discernment 
0% 0.04 8 0.04 x 8=0.32 


30% 0.00 80 0.00 x 80 = 0.00 
100% 0.36 12 0.36 x 12 = 4.32 
Total 100 4.64 
Average 4.64/100 


0.0464 


We might be tempted to conclude “there’s more 
discernment (0.0464) than accuracy error (0.002), 
therefore this forecaster is better.” Is that valid? What is 
the right way to combine these two numbers for a total 


“goodness” score? 


To answer that, it turns out there’s one more concept we 


need. 


a 
ANDERSON 
CRNSTAL BALLS et. 


“Seriously?! No one saw this coming?!” 


Discernment Error 


Consider the life of a forecaster in Antofagasta, Chile, 


where on average it rains only five days a year (for a 
grand total of 1.7 millimeters of annual rainfall!). At first 
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glance it seems easy to be a forecaster—just predict “no 


rain” every day. 


Although that forecaster would have low error, she would 
also be undiscerning. But wait... how could a forecaster 
ever be discerning in Antofagasta? To be discerning you 
need to make varied predictions. But reality isn’t varied, 
so any predictions that were varied, would necessarily be 
wrong! In a sense, there’s no “space” for discernment, be- 
cause there’s no variation to discern between. It’s not fair 
to ask the forecaster to be more discerning than the vari- 


ation that is actually available to discern. 


Compare that with forecaster in Portland, Oregon, USA 
where it rains 164 days out of the year—about 45%. And 
there’s no “rainy season”—it’s just chaotic. Here even just 
predicting 55% or 35% here and there could still be high- 
ly accurate but increase discernment. And a world-class 
forecaster has the space to create a significant amount of 


discernment. 


So it’s not quite fair to ask “How discerning is the fore- 
caster?” Instead we should ask “How discerning is the 
forecaster, compared with how much discernment is even 


possible?” 


The maximum amount of discernment possible, given the 


climactic average c, is*: 


c(1 —c) 


In general, the closer the climactic average is to 0% or 
100%, the less discernment there can be. The maximum 
possible discernment of 0.25 is available when the cli- 


mactic average is 50%, i.e. it’s a coin flip. 


2 For the mathematically curious: Maximum discernment hap- 
pens when the forecast is only 100% or 0%, and when it is 
completely accurate. If there were N predictions, the positive 
case will happen Nc times, and the negative case N(1—c) 
times. Discernment in the positive bucket is (1 — c)? and in the 
negative bucket (0—c)?=c?. The weighted average is 
wlNe(1 —c)? + N(1—c)c?]. Factoring out the common 
Nc(1—c) and canceling with +, we're left with 
ce(1—c)[(1-—c) +c] =c(1—c). 


In the 30% example, the maximum possible discernment 
is 0.30(1 — 0.30) = 0.21. Forecaster B’s discernment of 
0.04 is therefore not too impressive—plenty of room for 
improvement. Although still of course B is better than A, 


who had no discernment whatsoever. 


In the case of the desert, with a climactic average of 
5/365 = 0.013%, there’s only 0.0128 potential discern- 


ment available. 


In any case, this allows us to compute a metric that is 


comparable to error, but for discernment: 


[Discernment Error] = [Maximum Discernment] — 


[Discernment] 


That is, if you have no discernment, that’s another type 
of “error”—your forecast is lacking the descriptive power 
that would come from being maximally discerning. The 
more discernment you demonstrate, the less of this “dis- 
cernment error” you exhibit, and the better your forecast 
is. Just like the less accuracy error you have, the better 


your forecast is. 


Putting it all together: The Forecasting 
Error Score 


It turns out? you can simply add accuracy error to dis- 


cernment error, and arrive at a rigorous metric: 
3 See “Further Reading” below for the mathematical justification. 


[Forecast Error] = [Error] + [Discernment Error] 
Or, writing out each component: 


[Forecast Error] = [Error] + [Maximum Discernment] — 


[Discernment] 


Here’s a way to see why this math works: Every forecast- 
er’s baseline is to guess the climactic average. That will 
get you a total score of c(1 — c), because you have no ac- 


curacy error, but maximum discernment error. 


4o0f 5 


How to measure the accuracy of forecasts 


From there, forecasters will try to deviate from the cli- 
mactic average. The more they put themselves on the 
line, the less discernment error they rack up, however 
they also have to be right! The best forecasters outper- 
form the climactic-average (discernment) by more than 
the accuracy error they introduce. The overall score tells 
you who is better. Lower is better, since the score is a “to- 
tal error.” 


It is possible to do worse than the climactic average—to 
make guesses, but be wrong. You know that’s happening 


when the total error is larger than the baseline c(1 — c). 


Indeed, an alternate and equivalent scoring method di- 
vides Forecast Error by c(1 — c), to create a metric where 
1 means “equal to the baseline,” 0 means “perfect fore- 
cast,” and the amount greater or less than 1 indicates 
proportionally how much better or worse the forecast is 
from baseline. This has the virtue of the reader not need- 
ing to know the value of c(1 — c) in order to understand 
whether a forecaster is better or worse than the baseline, 


and by how much. 


While the total score is useful, the individual components 
of accuracy error and discernment are also useful be- 


cause they help you analyze what’s going on: 


Accuracy 
Error Discern Meaning Explanation 


t I) Failure You're not segmenting the 
population, and yet you’re 
still worse than guessing the 
climactic average. 

J J Useless You're only accurate because 

you’re just guessing the 

average. 

You're making strong predic- 

Again tions, so at least you’re try- 

ing, but you’re not guessing 
correctly. 


J t Ideal 


You're making strong predic- 
tions, and youre correct. 


Now that you know how to measure forecasts, it’s time 
for you to build some forecasting models. So go try to 
better understand your customers and prospects, and use 
this math to know whether iterations of your model are 


improving. 


Further Reading 


e Glenn Brier’s original paper proposing this method in 
1950, but without this three-component breakdown 
that was discovered by Allen Murphy in 1973. 

e Brier score on Wikipedia: A more formal explanation 
including the three-component breakdown (which 
are labelled and explained differently from my 
exposition, but which are mathematically identical). 

e Stein’s Paradox: An estimator that’s always better 
than the historical average, but in a way that 


apparently can’t be true. 
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by Jason Cohen on December 20, 2016 


“Building in public” is popular: How fun when strangers 
cheer you on! But isn’t competitive advantage ruined when 
competitors know your growth rate and steal your source 
code? 


"This just says you won't reveal anything about 
our nondisclosure agreement." 


What would happen if I forced you to put all your pre- 
cious, proprietary, secret-sauce code in a public Github 
repository? 


For one: You would be judged. Your peers would judge 
you by that code, with various definitions of “goodness” 
that includes everything from file and class organization, 
documentation, tests, eliminating your reliance on “secu- 
rity by obscurity,” tracking to-dos, and even the artful 
quality nebulously defined as “you know it when you see 


it. 1» 


i Famously said by US Supreme Court Justice Potter Stewart in 
1964 as his criteria for “obscenity,” though in 1972 he was part 
of a court that created a more specific three-part criteria; in 
1981 he said he “regrets having said what I said about obsceni- 
ty—that’s going to be on my tombstone.“ 


In this sense, building in public would force you to cre- 
ate quality, artful code. 


The force is as simple as it is universal: Ego—your desire 


to impress others. 


But... haven’t you lost your competitive advantage? Your 
competition can easily copy you, so you're screwed, 
right? 


You are therefore forced to build some other, non-techni- 
cal competitive advantage into your business. For exam- 
ple, if Etsy open-sourced their entire stack, would that 
make it easy for a competitor to overtake Etsy? No, be- 
cause Etsy has built a marketplace; the presence of buy- 
ers and sellers is their primary asset, not the software 
that drives the website or payments. Now that Facebook 
has open-sourced their entire data center infrastructure, 
does that make it easy for a competitor to overtake 
Facebook as a social network? No. The value in these 
companies cannot be replicated with money, which is ex- 


actly why those companies are valuable. 


A technical competitive advantage can be permanent in 
select cases, as in this case study of Amazon AWS. But 
this is rare; typically a technical advantage is a fleeting 
advantage, because most technical things can be replicat- 
ed. In fact, they're often replicated more quickly and 
cheaply than their original inventors, as proved by ob- 
serving that “first to market” often doesn’t equate to 
“winning the market.” 


Thus, building in public forces you to create perma- 


nent competitive advantages. 


“Building in public” doesn’t just mean software, however. 


What about the organization, or individual people? 
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If you publish the salary formula for everyone at your 
company, you no can longer compete for talent solely on 
compensation. You can’t kick in an extra $10k/year to 
win a critical employee with especially germane experi- 
ence. And it can be weird for everyone to know how 
much money everyone else makes. But for many people, 


it’s not weird, it’s fantastic. 


This forces you to build a company where great talent 
wants to work, where people are there for the culture 
and mission rather than as mercenaries. There are many 
frameworks that guide you in constructing such an envi- 
ronment, but in general they include Autonomy (i.e. 
freedom to explore, the control and empowerment to in- 
vent solutions to problems, and to prove the quality of 
those solutions through implementation, in exchange for 
accountability for the result), Growth (i.e. working on 
interesting puzzles, whether personal or professional, 
along a fulfilling journey), and Purpose (i.e. why should 
this company exist? Why is it worthwhile to bust ass to 


see it thrive, the culture, the values, the joy of working 


with others whom you admire). 


Here is a more comprehensive framework. 


Thus, building in public forces you to create a compa- 
ny with exceptional culture and an organization of 


empowerment and purpose. 


Writing code you can be proud of, constructing true per- 
manent competitive advantages, and creating a company 
culture that attracts and retains top talent for reasons be- 
yond legal bribery—are critical ingredients for creating 
durable companies whose growth and brand are unhin- 
dered by the onslaught of well-funded, hard-working, in- 
telligent upstarts that inevitably join any market big 


enough to be interesting. 


By building everything in public, you’re forced to do 
“what’s most valuable” as well as “what isn’t easily copy- 
able,” whether that means the quality of your workman- 
ship, the fortress of your business model, or a talent 


magnet. 


It’s still possible, and occasionally advisable, to operate in 
secret. But sunlight isn’t just the best disinfectant, it’s a 


great way to forge durability. 
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by Jason Cohen on May 10, 2017 


Fast, or Best? Choose your decision-making goal wisely, 
especially if you’re a natural perfectionist. 


“Before you see my grades, I'd like to remind you 
that I lowered expectations for the fourth 
quarter some time ago.” 


“Maximizing” means finding the best solution. It requires 
exploration and analysis to ensure “the best” option 
hasn’t been overlooked, and that we have confidence in 


our evaluation of the options. 


“Satisficing” means picking the first or easiest or least-ex- 
pensive option that satisfies the requirements. Preferring 
a faster decision to the best decision. It means not getting 
paralyzed by the pursuit of “perfect,” but as a result, 


rarely results in the very best solution. 


People naturally tend to be Maximizers or Satisficers, al- 
though it depends on the subject. For example, you might 


maximize your career, but satisfice your diet. 


Studies show that although Maximizers make better 
choices Satisficers enjoy their choices more, and of course 
spend less time and create less stress in making the 


choice. 


You must choose when to Maximize, and when to 
Satisfice. It might seem like Maximizing is best, especial- 
ly when you have teams of smart people who can do the 


maximizing. But often this is the wrong thing to do. 


Speed is one of the greatest competitive advantages, par- 
tially explaining why small up-starts are able to beat 
large incumbents. Speed requires satisficing, not maxi- 
mizing. Strength in startups comes from shipping, with 
benefits accruing today instead of theoretical benefits in 
future, using customer reaction and real data to decide 
your next move, rather than planning many moves 


ahead, which you can’t do anyway. 


Satisficing is also best for “sand” in the “Rocks, Pebbles, 


Sand” analogy, when deliberation is the enemy. 


Still, maximizing is best for some decisions, particularly 
decisions that should be made slowly. Your Product & 
Business Strategy is a slow obsession, best when taken 
from many angles, pounding it with devil’s advocacy, 
questioning every assumption, “sleeping on it” repeated- 
ly, repeating pieces of it to customers, strangers, anyone 
who might have a useful reaction. Strategy isn’t some- 
thing you want to change often, with entire teams and 


careers being built on top of it, so maximization is best. 


For the most important decisions, maximize with 
Binstack. For almost anything else, satisficing is likely 
wiser. Be careful; the same perfectionism that makes you 
a force to be reckoned with (and that drives your impos- 


tor syndrome) will lead you to maximize too often. 


For satisfying activities, define an objective threshold 
for “what is satisfactory.” When a metric is below its 
threshold, investment is warranted. Otherwise, you must 
not continue to invest in it, because definitionally it’s not 


valuable (enough) to deserve attention, whereas other 
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areas can leverage that investment, either because Pick the right goal for each decision, and communicate it 
they’re below their own thresholds, or because they are clearly. 


things to maximize. 
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by Jason Cohen on August 19, 2017 


Everyone wants change, but doesn’t want to change. 
Though inevitable, change is uncomfortable and exhausting. 
Manage it with kindness. 


Andrew Annett 
" @akannett 
Top workplace complaints: 


1) The way things are 
2) Change 


12:47 PM - Jun 1, 2015 


This plays out in many ways: 


1. Customers demand an improved UX, but they don’t 
want to learn a new UX, and they always complain 
when it changes. 

2. Team members want consistency but don’t want 
policies. 

3. Developers want to be more efficient but don’t want 
to change how they work. 


4. Strategy is ineffective if it’s constantly in flux, but a 


strategy that remains unchanged in the presence of 


new information is incorrect. 


The right choice is almost always “change.” This is be- 
cause change is a reaction to uncovering facts, getting 
smarter, or a shift in the external environment. Death 
awaits any organization that chooses the comfort of the 
familiar over the discomfort of change. 


Yet, though inevitable, change is uncomfortable and ex- 
hausting. Even we who relish change, who love bragging 
that “it’s hard but every day is different,” reach a break- 
ing point after years of adaptation and fake-gleefully ex- 


claiming that “failure is how you learn!” 


Yeah, but all this learning is fricking tiring. 


This is important for leaders to understand, if indeed 
“change is the only constant” as the insipid cliché goes. 
Even your most stoic, change-loving mortals sometimes 
need a break from change. Yes “it’s a marathon” but 
sometimes you need to walk a mile to catch your breath. 
Look for signs of burnout or decision-fatigue, and ad- 
dress it proactively. 


This is equally important for everyone in a startup, 
whether you manage others or not. Constant change can 
feel like management has no plan and no strategy. It 
takes careful consideration to distinguish between being 
rudderless and a culture of self-reflection and 


improvement. 


This is exacerbated by the fact that not all change is in 
fact for the best. Sometimes our attempt to solve a prob- 
lem makes it worse. Sometimes when we try to make 


code faster, we break it. The difference is that we can see 
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slow code objectively in the profiler and couch the code 
in unit tests and code reviews, and iterate before we ship, 
and revert to the previous version in the worst case; it’s 
not so easy when the change is happening to a whole 
team, or a major product release, or a cross-departmental 


strategic initiative. 


In fact, sometimes it’s objectively impossible to know 


ahead of time, and you have no choice but to place a bet. 


Even deciding what to change is hard. Successful compa- 
nies can stall out because they lose sight of the funda- 
mental reasons they earned success in the first place— 
the key insights and UX of the product, or the key culture 
and values that attracted their first hundred or thousand 
employees. But successful companies also stall out be- 
cause they're so dogmatic about their strategy or “non- 
consensus but correct” ideas that when the world 
changes around them, or scale breaks their previously- 
correct notions, they fail to adapt. It is not generally true 
that “what got us here will get us there,” and that means 


deep change is required. 


There’s a mindset that everyone can use to address all of 


these difficulties: 


Be kind. 


Maybe don’t judge too harshly if your organization tries 
to improve one thing and ends up breaking another, or 
where the organization takes too long to implement 
change. Maybe don’t judge too harshly if the person to 
your left needs to work on something easy for a few 


sprints or take a vacation. 


Edison had to try thousands of materials before finding 
the one that make lightbulbs practical. Would you have 
judged him for “thrashing?” Invention is often 


frustrating. 


You should judge harshly if nobody is thinking about this. 
If nobody cares whether there’s change or not, if there’s 
no rhyme or reason to the company strategy, if everyone 
is expected to act and feel happy and productive all of 
the time, then you should definitely judge. An organiza- 
tion that isn’t striving to improve, will rot and 


disintegrate. 


There are no straight paths in life or startups. All we can 
do is keep being introspective, and keep attempting the 


right sort of change. 
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by Jason Cohen on August 22, 2017 


“MVP” is a selfish process, abusing customers so you can 
“learn.” SLC is an alternate philosophy that results in fast, 
validated learning, that customers love. 
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“What was initially thought to be a simple process is 
in fact an incredibly complicated, intricate, and complex 
system that I've codified and organized into a few 
easy-to-follow rules that are more difficult 
to implement than you’d think.” 


Product teams have been repeating the MVP (Minimum 
Viable Product) mantra for a decade now, without re- 
evaluating whether it’s the right way to maximize learn- 


ing while pleasing the customer. 


Well, it’s not the best system. It’s selfish and it hurts cus- 
tomers. We don’t build MVPs at WP Engine. 


The motivation behind the MVP is still valid: 


1. Build something small, because small things are 
quick and inexpensive to test. 

2. Get it into the market quickly, because real learning 
occurs only when real customers are using a real 
product. 


3. Trash it or hard-pivot if it’s a failure, or invest if it’s a 


seedling with potential. 


MVPs are great for startups and product teams because 
they maximize so-called “validated learning” as quickly 


as possible. And while customer interviews are useful, 


you learn brand new things when a customer actually 
uses the product. But MVPs are a selfish act. 


The problem is: Customers hate MVPs. Startups are en- 
couraged by the great Reid Hoffman to “launch early 
enough that you’re embarrassed by your v1.0 release.” 
But no customer wants to use an unfinished product that 
the creators are embarrassed by. Customers want great 


products they can use now. 


MVPs are too M and rarely V. Customers see that, and 
hate it. It might be great for the product team, but it’s 
bad for customers. And ultimately, what’s bad for cus- 


tomers is bad for the company. 


Fortunately, there’s a better way to build and validate 
products. The insight comes by honoring the utility of 
MVPs (listed above) while giving just as much considera- 


tion to the customer’s experience. 


In order for the product to be small and delivered quick- 
ly, it has to be simple. Customers accept simple products 
every day. Even if it doesn’t do everything needed, as 
long as the product never claimed to do more than it 
does, customers are forgiving. For example, it was okay 
that early versions Google Docs had only 3% of the fea- 
tures of Microsoft Word, because Docs did a great job at 
what it was primarily designed for, which is simplicity 


and real-time collaboration. 


Google Docs was simple, but also complete. This is de- 
cidedly different from the classic MVP which by defini- 


73 


tion isn’t complete (in fact, it’s “embarrassing”). “Simple” 
is good, “incomplete” is not. The customer should have a 
genuine desire to use the product, as-is. Not because it’s 
version 0.1 of something complex, but because it’s ver- 


sion 1.0 of something simple. 
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It is not contradictory for products to be simple as well as 
complete. Examples include the first versions of 
WhatsApp, Snapchat, Stripe, Twilio, Twitter, and Slack. 
Some of those later expanded to add complexity 
(Snapchat, Stripe, Slack), whereas some kept it simple as 
a permanent value (Twitter, WhatsApp). Virgin Air and 
Southwest Airlines both started with only a single route 
—small, but complete. 


The final ingredient is that the product has to be lovable. 
People have to want to use it. Products that do less but 
are loved, are more successful than products which have 
more features, but that people dislike. The original, very- 
low-feature, very-highly-loved, hyper-successful early 
versions of all the products listed in the previous para- 
graph are examples. The Darwinian success loop of a 
product is a function of love, not of features. 


There are many ways to generate love. “Minimum” and 
“viable” are definitely not among those ways. The cur- 
rent-in-vogue way is through design: Elegant UX com- 
bined with delightful UI. But there are other ways. The 
attitude and culture of the company itself can generate 
love, such as Buffer’s blog with its surprising trans- 
parency or MeetEdgar’s blog genuinely helping entrepre- 
neurs or HubSpot’s blog which early on was at least as 
instrumental to their customers’ success as the actual 
product. Another way is through a deep connection to 
the psyche and work-style of customers, like Heroku who 
broke with marketing tradition by filling the homepage 
with command-line examples instead of benefit-state- 
ments, thereby connecting instantly with their geeky tar- 


get customer: 


Create 


Deploy 


Work 


Heroku Garden Transition Info 


See this article for many more examples of how to gener- 


ate love. 


From this reasoning, years ago I named what I believe is 
the correct alternative to the MVP: Simple, Lovable and 
Complete (SLC). We pronounce it “Slick.” As in: “What’s 


the ‘Slick’ version of your idea?” 


Another benefit of SLC becomes apparent when you con- 


sider the next version of the product. 


A SLC product does not require ongoing development in 
order to add value. It’s possible that v1 should evolve for 
years into a v4, but you also have the option of not in- 
vesting further in the product, yet it still adds value. An 
MVP that never gets additional investment is just a bad 
product. A SLC that never gets additional investment is a 


good, if modest product. 


Although not called SLC, there’s a popular meme in prod- 
uct circles that neatly encapsulates the idea of SLC in a 
diagram: The Modes of Transportation example! from the 


Spotify product team: 
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Your customers hate MVPs. Make a SLC instead. 


1 Critics correctly point out that Tesla did not follow this system 
—they didn’t make a skateboard and then a bike and finally a 
car, they just made a car. This is a valid critique, when you 
know that the only product goal is “car.” Indeed, there isn’t a 
SLC version of a car—it’s a leap of faith. Still, human society 
did evolve someone like this, with basic wheeled objects, later 
a bike, and still later a car. Also this model is especially useful 
when your startup set out to make a simple product, not even 
knowing that a car could exist. Software companies often do 
evolve this way; the main text gives a real-world example. 


HOW TO BUILD A MINIMUM VIABLE PRODUCT 
NOT LIKE THIS 


LIKE THIS 


image by blog.fastmonkeys.com original ides: spotify product team 


A skateboard is a SLC product. It’s faster than walking, 
it’s simple, many people love it, and it’s a complete prod- 
uct that doesn’t need additions to be fun or practical. At 
the same time, you can evolve the skateboard by adding 
a stem and handlebars, to create a scooter—only slightly 
less simple, and definitely loveable and complete. Next, 
you could grow the wheels, add a seat and some gears, 
and you have a bike. Again, less simple but now you have 
a product with massive benefits of speed, distance, and 
energy-efficiency. 


Zooming into one of our examples above, Snapchat took 
an SLC progression similar to the transportation 
metaphor. The first iteration of the product was a screen 


where tapping anywhere took a picture that you could 


then send to someone else, at which time it disappeared. 
No video, no filters, no social networking, no comment- 
ing and no storage—simple, yet Lovable and Complete, 
as evidenced by its massive adoption. The insight of “no 
storage” was critical, but many people have theorized 
that the simplicity of the interface was also critical. The 
very fact that it was simple, while not sacrificing love- 
ability or completeness, caused its success. 


Later they added lots of stuff—video, filters, timelines, 
even video cameras inside sunglasses. It’s OK for prod- 
ucts to become complex. Starting out SLC does not pre- 


clude becoming complex later. 


With SLC, the outcomes are better and your options for 
next steps are better. If it fails, that’s OK; it’s a failed ex- 
periment. Both SLCs and MVPs will sometimes produce 
that result because the whole point is to experiment. But 
if a SLC succeeds, you’ve already delivered real value to 
customers and you have multiple futures available to 
you, none of which are urgent. You could build a v2, and 
because youre already generating value, you have more 
time to decide what that should look like. You could even 
query existing customers to determine exactly what v2 
should entail, instead of a set of alpha-testers who just 
want to know “when are you going to fix this broken 
thing?” 


Or, you can decide not to work on it. Not every product 
has to become complex. Not every product needs new 
major versions every two quarters. Some things can just 
remain simple, lovable, and complete. 


Ask your customers. They'll agree. 
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How repositioning a product allows you to 8x its price 


by Jason Cohen on June 5, 2018 


You can charge much more than you think, if you reposition 
your value-proposition. Here’s how. 


ANDERSN 
“T like it, but I’m looking for more of a 
status symbol. Any way you can double 
the price?” 


Pricing is much more about positioning and perceived 
value than it is about cost-analysis and unconvincing ROI 
calculators. Let’s see how repositioning can result in a 


much higher price for the same product. 


You’ve created a marketing tool called DoubleDown that 
doubles the cost-efficiency of AdWords campaigns. You 
heard that right folks—as a marketer, you can generate 
the same impact, the same number of conversions, the 
same quality of sales leads, but with half your current ad- 
spend. Wonderful! Who doesn’t want higher ROI. 


What can you charge for this tool? Well, the customer 
will save a certain amount of money on ad-spend; surely 
you cannot charge more than that. Let’s say you can 
charge 25% of the savings and still find many willing 


customers. 


Here’s what your sales pitch looks like to a customer who 
spends $40,000 per month on AdWords: 


Halve your spend! 
$40,000 


$30,000 


$20,000 


$10,000 


Before After 


Great deal! The VP of Demand Gen will be able to boast 
to the CMO that she saved the company $15,000/mo 
even after paying for DoubleDown, and you’re raking in a 


cool $5,000/mo. Everyone’s happy! 


Now let’s see how to charge eight times as much mon- 


ey for the same product. 


Marketers have a paramount goal: Growth. Even indirect 
marketing like brand, events, and PR have the indirect 
goal of supporting growth. In the case of DoubleDown’s 
customers it’s direct: Growth through lead-generation 
through AdWords. 


Increasing growth is much more valuable than de- 
creasing cost. To see why, consider the following two 


scenarios: 


1. CMO reports to the CEO: I was able to reduce costs 
20% this year. 
The CEO is happy. The CEO’s follow-up question is: 


How will we use those savings to grow faster? 
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2. CMO reports to the CEO: I was able to increase 
growth by 20% this year, but it also cost us 20% 
more to achieve. 

The CEO pumps her fists amongst peels of joyous 
laughter. The value of the company increases non-lin- 
early. The additional revenue growth more than pays 
for the additional marketing cost that generated it. The 
CEO’s follow-up question is: Can we spend even more? 


How can we ensure this happens again next year? 


It’s always 10x more valuable for a business to grow 


faster than it is for the business to save money. 


This insight points us to an alternate pitch for 
DoubleDown. It’s not about spending less for the same 
amount of growth, it’s about spending more to create 
more growth. 


Continuing our example, suppose the customer generates 
200 quality sales leads per month from their $40,000/mo 


spend. The sales pitch changes as follows: 


You're paying $200/lead right now, yielding 200 leads 
per month. Using DoubleDown, you can double the num- 


ber of leads you’re generating, still at a cost of $200/lead: 


Double your leads! 
400 + . . 


300 


200 


100 


Before After 


The customer is willing to spend $40,000 to generate 
200 leads, and therefore is happy to spend $80,000 to 
generate 400 leads. It doesn’t matter how much of that 
$80,000 is going to AdWords versus going to 
DoubleDown. The goal is not to “save money on 
AdWords,” but rather to “generate more growth at a simi- 


lar unit cost.” 


In the “saves money” pitch, the value was $20,000, and 
the customer needed to keep 75% of that value-creation. 
Whereas in the “generate growth” pitch, the value is 
$40,000, and the customer is happy to pay 100% of that 
value-creation to a vendor. Both the amount of value 
created, and the percentage of value the customer is 
willing to pay, is a multiple higher for the “growth” 


pitch versus the “save money” pitch. 


So the next time you want to formulate your product as a 
way to “save time” or “save money” or “be more efficient” 


.... don’t. 


Instead, figure out how your product creates value in the 
way your customer already measures value, and position 
your product as a way to accomplish that. 
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When should a decision be fast, or slow? 


by Jason Cohen on September 5, 2018 


Decisions should usually be made quickly, to accelerate 
action and learning. But sometimes it really is smarter to 
take your time. Here’s how to decide. 


or staunchly opposed. But rest assured, 
however I vote, it will be staunchly.” 


We all know that startups should make decisions quickly. 
Fast decisions leads to rapid action, which accelerates the 
loop of production and feedback, which is how you out- 
pace and out-learn a competitor, even one that already 


has a lead. 


But some decisions should not be made in haste, like a 
key executive hire, or pricing strategy, or whether to raise 
money, or whether to invest millions of dollars in a new 


product line, or whether to enter a new market. 


How do you know when your current decision should be 
made slowly: contemplative, collaborative, deliberate, 


data-driven, even agonizing? 


The following scorecard will help you know whether it is 


wiser to go slowly: 


Yow’ll make a much smarter decision later 
Today we know the least that we’re ever going to 
know. Often, waiting six months doesn’t automatically 


give us more information, but sometimes it does. If we 


haven’t launched a product yet, we’re in no position to 
decide what the next few features should be; we 
should launch and earn experience through customer 
interactions. We’ll then be much smarter about an- 
swering that question. So, don’t waste time even 
thinking about it now, and certainly don’t make any 
firm decisions, when you know you'll make a better 


one later. 


Can’t undo 

This is the classic one-or-two-way door delineation. If 
you can’t easily undo the decision, it’s worth investing 
more effort into analyzing the likelihood of the up- 


sides and risks. 


Huge effort 

Some things take less time to implement than to esti- 
mate or to debate (i.e. Sand). It might take two engi- 
neers one week to implement something, but a few 
debates and a time-boxed research project could in- 
volve an entire team for a week. This is a reason why 
small teams without process can produce results faster 
than larger teams with process. If the effort to imple- 
ment the decision is smaller than the effort to make a 
decision, just knock it out. But if you’re deciding on a 
path that could take six months to measure results 
from, taking time up front to research is wise. 


No compelling event 

If the status quo isn’t bad, there might not be a reason 
why a decision should be made quickly, or at all. 
Without time-pressure, it’s justifiable to spend more 
time on the decision. Conversely, time-pressure means 
the more time you spend deciding, the less time you 
have for implementation and unanticipated problems, 
so youre actually adding risk by dragging out the 


decision. 
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“We're taking a wait and see and then do 
nothing approach.” 


Not accustomed to making these kinds of decisions 
Online marketing teams are accustomed to throwing 
creative things at the wall, because that’s the day-to- 
day reality of their job. Because they’re good at it, they 
don’t waste time hang-wringing over whether or not 
to try an advertising campaign on the latest social me- 
dia platform, or which headlines to try; they just do it. 
Conversely, most organizations have no experience 
with major decisions like pricing changes or acquisi- 
tions, and most founders have no idea how to hire a 
great executive, or how to decide whether to invest 
millions of dollars in a new product line as opposed to 
‘Gust throwing something out there and iterating” as 
was the correct path at the start of the company. When 
the organization has never made this type of decision 
before, the decision is at great risk, and being more 
deliberate with research, data, debate, or even outside 


advice, is wise. 


Don’t know how to evaluate the options 

Even after generating the choices, does the team un- 
derstand how best to analyze them? If the company’s 
strategy is clear and detailed, if relevant data is at 
hand, if it’s clear what your goals are, if the deciding 
team has confidence, then the decision could be easy 
and fast; if these things are absent, perhaps more de- 


liberation is needed to clarify those things. (Another 


technique is to use Fermi Estimation to simplify the 
evaluation, or Binstack to find the one best option 


among many.) 


Can’t measure incremental success 

After the decision is made and implementation begins, 
can you objectively tell whether things are going well? 
If yes, it is easy to course-correct, or even change the 
decision, in the presence of reality. But if progress will 
be invisible or subjective, such that you will sink per- 
son-years of time into the implementation before 
knowing how things are going, it’s worth spending 
more effort gaining confidence in the path you’ve 
selected. 


Imperfect information 

Buying a house is nerve-racking, mostly because it is 
likely the most expensive and difficult-to-undo pur- 
chase of your life, but also because you know so little 
about the goods. What does the seller know but isn’t 
telling you? What will you not discover until you’ve 
moved in, or a year later? Often it is impossible to get 
the data or research you need to make an objective 
decision. When this is the case, it is sometimes wise to 
spend extra time gathering whatever information you 
can, maybe investing in reports or experts (which is 
what you do with a house). Or you could look at it the 
opposite way: If it’s impossible to get objective data 
informing the decision, then don’t spend lots of time 
debating subjective points; just make the decision 
from experience and even gut-check, because we just 
said that’s all you have to go on anyway, and experts 


are often wrong. 


Decision requires multiple teams who haven’t 
worked together before 

At WP Engine we’re extremely collaborative across 
teams. The benefit is that we work together for a com- 
mon goal, taking care of the needs of support, sales, 
marketing, engineering, product, and finance, rather 
than solving for one department’s goals at the expense 
of another. But this also can make decisions more dif- 


ficult, because finding a good solution is complex, of- 
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ten requiring compromise or creativity which requires 
time to be realized. This effect is amplified if the 
teams (or team members) haven’t worked together be- 
fore, and thus have less rapport, common language, 
and common experience. In that case, give the deci- 
sion more time to breathe and develop, because really 
youre giving people the time to build relationships and 
discover great solutions, and that in itself is a benefit 
to them and your organizational intelligence, which is 


a long-term benefit worth investing in. 


“Ma’am, there’s really nothing I can do until they’ve been 

missing for at least 24 hours. In my experience, the best 

thing to do is just leave it alone and they'll come home, 
wagging their tails behind them.” 


Actually this isn’t a scorecard, because important deci- 
sions aren’t a Cosmo Quiz. Don’t use this as a rubric; 


don’t score it 1-5 and add it up with a spreadsheet. 


Rather, honestly answer these questions, and by the time 
you’re through, you’ll have a good sense of whether a 
light touch, quick decision is fine (which is the default 


answer), or whether you should take more time. 


And, depending on which pieces are problematic, you'll 
have a guide for what needs to be done next. 


For example, if “Can’t undo” is a big problem, can you re- 
think the solution so that it can be undone, maybe by ap- 
plying it only in one segment, or investing more time in 
planning or preparation, or creating a disaster recovery 
plan, or splitting up the decision so that part of is is 


undoable? 


Or if “No compelling event” is a problem, maybe the best 
answer is to “not decide,” i.e. don’t spend time on this 
right now, since you don’t have to. Some people will be 
disappointed in the lack of a decision, but it’s better to 
honestly state that “we can’t figure out the answer right 
now” than to make a rash decision that does more harm 
than good, or to invest time in a decision that doesn’t 
need to be made, at the expense of work that does need 
to be done. 


I hope this helps you make the right decisions, in the 
right way. 
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by Jason Cohen on February 23, 2019 


We’re obviously in no danger of 
arriving at consensus.” 
—wWarren Buffet 


Time is notoriously difficult to synchronize in distributed 
systems. Many algorithms require—or have better char- 
acteristics when—any number of independent replicas 
achieve a single global time-ordering. In particular, it is 
useful to know that if two events are separated by more 
than a few seconds in real-time, that they will be correct- 
ly ordered, despite replicas having unsynchronized physi- 


cal clocks. 


However, operating system clocks are undependable and 
can even move backward. Furthermore, we don’t want to 
rely on a “master” or central server. Replicas should be 
able to arbitrarily join or leave a collective without an- 
nouncement, operate independently, and communicate 


with each other peer-to-peer, in any pattern. 


Common methods to solve the problem either involve 
custom infrastructure to achieve tight bounds on the be- 
havior of operating system clocks (e.g. Google 
TrueTime), or some sort of distributed logical clock such 
as Vector Clocks. The former is unavailable to most of us, 
especially if we want our code to run in browsers or lap- 
tops or non-Google data centers. The latter suffers from 
unbounded memory requirements and still results in 
“conflicts” which are resolved arbitrarily or not at all. 


Here we present an algorithm and Golang implementa- 
tion that meets the objectives, with no assumptions about 
the behavior of operating system clocks, without vector 


clocks, and without central coordination. 


Objectives & Definitions 


It’s useful to establish a few definitions and conventions 


before describing how we achieve a solution: 


e RT—Real Time—theoretical omniscient “actual” 
time. Imagine it as arbitrarily-high-precision UTC 
time exactly synchronized everywhere. This is not a 
quantity that any replica can compute; it is for 
algorithmic discussion only. 

e PT—Physical Time—the time reported by the 
operating system. This is the only notion of “time” 
that a replica can access. 

e LT—Logical Time—our implementation of “time,” 
with the properties outlined above and precisely 
defined below. 

* XT{expression} means “the type of time xT, 
applied to the replicas or events in the expression.” 
For example, pT{ P<Q } means “the Physical Time 
on the replica P is less than the Physical Time on 


replica Q.” 


With this terminology, we can define our assumptions 


and our requirements for LT precisely: 


1. PT is arbitrary. It can move backwards, it does not 
produce unique values, it might update only rarely, it 
is never synchronized between two replicas, it can be 
ahead of or behind RT by an arbitrary and variable 
amount. The only thing we know is that, over a long 
duration, it generally increases. 

2. LT always increases. Whenever LT is requested, it 
will have increased since the previous request 
(regardless of PT’s behavior). 

3. If B happens after A in real-time, i.e. RT{ B>A }, but 
A and B happen on different replicas: 

a. Correctly ordered in LT when sufficiently 
time-separated in RT. If rt{ B-A } > e, where 


e is asmall, bounded constant, then LT{ B>A } 
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also. 
b. Arbitrary LT order if close together in RT. If 
RT{ B-A } < e, then there’s best-effort for 


LT{ B>A }, butit could be LT{ B<aA }. 

c. e is small and bounded. e must be small (e.g. 
1-2 seconds), and be a constant, not proportional 
to some state or configuration. e is the time 
window inside which we accept events that are 
incorrectly ordered, so we require this window to 
be small. 

d. “Happened-before” relation is always 
correctly ordered in LT. If A happens, then the 
two replicas communicate, then B happens, then 
it is always true that LT{ B>A }, evenif RT{ B- 
A}<e. 

4. Skewed PT on one replica doesn’t skew the 
behavior of the collective. If one replica’s PT differs 
significantly from RT, it should still participate 
properly with the collective; for example it shouldn’t 
automatically “win” due to the late-skewed PT, or be 
unable to write a change due to early-skewed PT. 

5. LT uses constant memory. Use a fixed number of 
bits to represent LT, regardless of factors like the 
number of replicas. (Unlike Vector Clocks.) 

6. Decentralized, without “join” or “leave” events. 
No masters, no central API. Peers can communicate 
in arbitrary patterns. Peers never need to announce 
their joining or leaving the collective. There is no list 


of peers. 


It may sound impossible to achieve the goals of LT given 
that RT is inaccessible, PT lacks all the properties we 
need, and replicas aren’t synchronized. It’s fun to see 


how it can in fact be achieved. 


LT inside a single replica 


It might seem trivial to create LT within a single replica, 
but even that requires an algorithm. Typically the operat- 
ing system is used for PT, but that can move backwards 
(e.g. with NTP with leap-seconds, with manual settings), 
and is non-unique (e.g. asking for the time in rapid suc- 


cession often yields the same value from the operating 
system, due to CPUs being faster than the precision of 
time), and can drift significantly far away from RT (due 


to various causes as in these examples). 


PT will generally increase in the long-run, but we have to 
“smooth over” the bumps where it is stagnant or decreas- 
ing. To do that, we store the “last-seen PT,” and also a 
counter, and use the following algorithm (plus a mutex 


for concurrent access): 


func GetLT() { 
if PT Current > PT_LastSeen { 
PT_LastSeen = PT Current 
Counter = 0 
} alee: 
Counter = Counter + 1 


} 


return {PT_LastSeen, Counter} 


} 


LT is ordered first by PT LastSeen and then by 
Counter. So, as long as PT increases, we use it, staying 
hopefully somewhere near RT (more on this later), but if 
PT does not increase, the Counter ensures that LT still 


increases. 


In the following simulation from our golang implementa- 
tion, we can see how LT monotonically increases even 


when PT doesn’t: 


LT monotonically increases regardless of how crazy PT is 


100 


LT (Logical Time) 
Ri. 


30 60 
RT (Real Time) 


The use of a counter also means the precision of PT is not 
important. Therefore, a performance optimization is to 
use a recurring timer to update a thread-safe global vari- 
able with PT 1-4 times per second, rather than invoking 


the much more expensive and blocking operating system 
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call to retrieve PT every time we compute LT. With this 
optimization, we achieved tens of millions of invocations 
per second even in a Javascript implementation on a typ- 


ical laptop. 


“Happened-Before” relation between 
replicas 


Suppose an event A happens on a replica P and then P 
communicates with another replica Q, sending that 
event. Afterwards, an event B happens on Q. We want to 
be certain of LT{ B>A }, even though As LT was gener- 
ated on P and B’s LT was generated on Q. (See diagram 


below) 


The PT of P and Q will differ, and could differ in either 
direction. If pt{ Q>P }, then we'll get LT{ B>A } natu- 
rally, because the PT component of Q is already ahead of 


the PT component of P That’s the easy case. 


In the other case that pT{ P>Q }, we havea problem. In 
the example below, P’s PT is one minute ahead of Q’s. 
After P sends Q event A with LT{ 71.0 }, Q’s LT is still 
far behind, which means when event B happens one sec- 
ond later, it is LT{ 13.0 }, resulting in the problem 


LT{ A>B } eventhough RT{ aA<B }. 


, in. | 
LT (70.0) 72.0 
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~ 
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Problem: B’s LT isn’t later than A’s LT, because they’re on dif- 
ferent servers. 


To fix this, we simply set both components of Q’s LT 
equal to P’s. Q will know to do this, because when P com- 
municates with Q, it transmits its current value of LT. The 


following algorithm ensures Q will end up with a strictly- 


larger LT: 


func UpdateLTFromPeer( LT Peer ) { 

// Operate on current LT 

LT_Local = GetLT() 

// Take the latest LT 

if LT Peer > LT Local { 
LT_Local = LT Peer 

} 

// Ensure strictly larger than any previous LT 

Counter = Counter + 1 

// This is the new local LT 

SaveState (LT Local) 


A side effect is that Q’s PT_LastSeen will be ahead of its 
own PT, but that’s fine because Q will just use its Counter 
until its own PT catches up. Meanwhile, pT{ B>A } is 


guaranteed, as the diagram now shows: 


p & 


PT 70 7 72 


eT. LO [| [2 (3 
LT 10.0 =f1.0 7.0 TI. 


Solution: Q appropriates P’s LT, because LT{ P>Q } 


Further discussion and a proof of correctness can be 
found in the paper that invented this method, in which it 
is called a Hybrid Logical Clock (HLC). 


Using Skew to fix the future 


Although the algorithm above satisfies many of the re- 
quirements of LT, it violates the requirement that e be 


small and bounded. 


To see why, let’s extend our example to consider what 
happens with subsequent events on P and Q. In particu- 


lar, P generates an event C soon after B (in RT), and then 
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Q generates an event D about thirty seconds after that: 


30 
P (A) S20cOVnaS 
t ' t n ee 
FT: 70 71 72 «#73 #974 lof 10S 
LT 2.0 720 73.0 (140) [04.0 [05,0 
"71.0" ~ 
del 3O 
Q S2OOVGS 
+ t t { a rrrrstrrrrr) —+—___j_—. 
PT Te) I! Im 13 14 wy AS 
LT [00 #.0 7.0 7. 72 "13, WAY 


Problem: PT clock-skew breaks LT ordering of future events C 
and D, even though they differ by 30 seconds in RT 


We've highlighted the problem: LT{ c>pD }, even though 
RT{ C<D }. The cause of the problem is displayed in the 
diagram: Because Q’s PT is so far behind P’s, Q has to use 
its Counter to increment its LT, but meanwhile P is in- 
crementing its LT using its PT. In fact, every event on P 
during the minute after P communicated with Q will 
have an LT greater than every event on Q during the 
same interval, regardless of their ordering in RT. This is 
the condition described in our original LT goals where 
events can be mis-ordered in LT if they happen closer 
than a duration e. The trouble is, e is too big (it’s one 
minute in this example) and it’s not bounded (it could 


just as easily be one hour). 


This situation remains even after Q’s PT catches up with 
the synchronization event. P’s LTs will always have a larg- 
er PT component than Q’s LTs, and thus P’s events (with- 
in any one-minute time window) will always look like 
they are later than Q’s in LT, regardless of their order in 
RT 


One solution is to mandate small PT clock skews, which 
in turn mandates that e is small. For example, a basic 
NTP service can keep clocks synchronized to within tens 
of milliseconds. In this case the effect in our example 
would still exist, but only inside a tiny time window of 
tens of milliseconds, not a full minute. That would be 


acceptable. 


However, our implementation does not assume control 
over PT. A replica might be a browser or laptop that we 
don’t control, or a virtual machine that isn’t running NTP 
So, we need an algorithmic extension that eliminates this 


problem of PT clock skew. 


The solution is for Q to compute the PT clock skew, and 
use that as an offset to its native PT to stay reasonably 
up-to-date with P This cannot be done precisely, because 
of the non-measurable and often-variable communication 
transmission delay between P and Q, and because PT 
isn’t dependable, but it turns out being precise is not 


necessary. 


When P communicates its LT to Q, Q computes 
s = PT{ P-Q } as the “skew.” As the diagram below il- 
lustrates, s will always under-estimate the actual clock 
skew, because it’s not taking transmission-time ad into ac- 
count. In our example here, the actual clock-skew is 60, 
but the computed skew is 59 due to the transmission de- 


lay d=1. 


When s is positive, Q saves s. The next time PT is com- 
puted, Q uses PT{ Q } + s as physical time. This means 
Q’s idea of physical time is now only a behind P This 


nullifies the problem in our example: 


p (A) spends 
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Solution: Use an approximation of real clock skew to reduce 
the time-window of out-of-order LTs 


If s is negative, it is ignored; this ensures that clocks 


that are already ahead do not get even further ahead. 
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Although in practice d is not measurable and fluctuates, 
it is always non-zero, and rarely larger than a few sec- 
onds. It is proportional to network transmission time, not 
proportional to PT clock skew or any other system state 
or configuration. Therefore we can say that s always un- 
der-estimates skew, and by a bounded amount on the or- 
der of the replica’s communications delay (i.e. 1ms inside 
a data center, 100ms across a country, or 1000ms across 
the world). 


Finally, observe from the diagram that the time-window 
in which this problem can occur has been reduced to just 
1 second, i.e. reduced to a. Indeed, e from our LT goals 
is exactly a theoretically, and on the order of a practi- 
cally, which we just said was less than a few seconds. 
Thus, we have achieved the objective that e be a bound- 


ed constant. 


What happens when all replicas’ PTs are in fact well-syn- 
chronized, e.g. with an error less than a, which is easily 
achievable with well-known algorithms like NTP or mod- 
ern phones and laptops that synchronize their clock with 
GPS? Then the computed skew will be less than zero. To 


see why, consider that 


[computed skew] = [real skew] - d, but in this hy- 
pothetical, [real skew] might be 50ms whereas d is 
typically greater than that. Negative computed skews are 
ignored, thus we'll always have s = 0. 


Although this is not a specific requirement on the behav- 
ior of LT, it does satisfy an intuitive desire for skew-cor- 


rection to vanish when it isn’t needed. 


If a replica’s PT is substantially earlier than RT, it will de- 
velop a large forward skew, neutralizing the problem. If a 
replica’s PT is substantially later than RT, all other repli- 
cas will develop a skew that aligns with it. Therefore, we 
achieve the objective that significant skews in either di- 
rection don’t adversely affect operation of those replicas 


or of others. 


Still, a replica with a large pt-rRT will create a large 
skew value for the whole group, with the legal but unde- 
sirable effect that pt + s differs significantly from RT. 
When that happens, it’s important that skews don’t con- 
tinue to creep up, with each replica edging the others for- 
ward. A non-zero value of a helps; in our implementa- 
tion we add another 500ms to the effective value of a to 


ensure this effect. 


Simulations 


The following simulations were generated from our 


Golang implementation. 
Convergent LT, with staggered PT 


With replicas starting with PT staggered every 10-sec- 
onds, one-way-synchronizing a random pair once per sec- 
ond, they monotonically increase and eventually con- 


verge on the one with the latest LT. 


One-way sync between random pairs; everyone monotonically increases and eventually synchronizes 


LT (Logical Time) 


T 
0) 30 60 90 


RT (Real Time) 


Convergent LT, with variable-rate PT 


With each replica’s PT clock running at a different rate 
relative to RT, one-way-synchronizing a random pair 
once per second, all replicas keep converging close to the 
one with the latest LT, i.e. the fastest clock. 
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One-way sync between random pairs; everyone monotonically increases and eventually synchronizes 


150 


90 


LT (Logical Time) 
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RT (Real Time) 


Far-Future replica joins, then leaves 


A replica with a far-future date joins; all replicas con- 
verge on the new far-future LT by one-way-synchronizing 
a random pair once per second. The “bad” replica then 
leaves the collective. The remaining replicas have large 
skews, which should not change. In particular, they 


should not “creep up” in skew. 


One-way sync between random pairs; everyone monotonically increases and eventually synchronizes 


150] 


LT (Logical Time) 


J 
30 pe 
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Problems 


LT at start-up 
When a replica first starts up, it will have an LT that is 
likely to need skew-correction. It should “fix” its LT prior 


to using it in a meaningful way. 


One fix is to communicate with any other replica; this 


will bring it up to speed and set an appropriate skew. 


Another fix is to persist the LT state between runs of the 
replica. In particular, saving the skew value. However, 
this is not as good as communicating with a live replica, 
because the behavior of PT or the collective value of the 


skew might have changed since the previous run. 


Using LT without doing those things is still legal and self- 
consistent, but will generate events that will appear to be 
older than they actually are, relative to other events be- 


ing generated by other replicas around the same RT. 


Anti-Objectives 


The following are not goals. In some cases the algorithm 
gives best-effort to achieve them anyway. In some cases 
there are things the library user can do to opt-into having 
that goal, possibly at the expense of another goal or 


constraint. 


Uniqueness 
The algorithm above creates locally-unique values (i.e. 
monotonically-increasing), but not globally-unique (i.e. 


two replicas can generate the same LT). 


Uniqueness can be useful because it allows LT to also 


serve as a “name” of an event in logs or databases. 


It’s easy to add uniqueness. Just add more (least-signifi- 
cant) bits to the LT structure. Set them equal to some- 
thing unique to a replica. This can be a replica ID that is 
unique in the world, since the rest of the Time compo- 
nents will never be generated again on that replica. Or it 


can be a sufficiently large number of random bits. 


It may seem like collisions are already unlikely, however 
they are common under certain assumptions, namely if 
PT is coarsely updated and a is very small. Consider the 
example above, but rather than d=1, assume a fast net- 
work where d=0.001 but a PT source that updates only 
once per second (on a timer, say). Once P and Q share LT, 
they will be identical at the same point in RT, and stay 
synchronized thanks to s. So they will likely collide if 


both generate an LT inside the same RT second. 


Keep PT close to RT 
It’s nice if PT stays close to RT, but it is not a 


requirement. 
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You can achieve this, in fact making the difference 
bounded, if you disable skew. This keeps PT close to RT 
but results in a potentially large e, and thus you get mis- 
ordered events. If you accept this trade-off, you can en- 
sure PT never strays too far from RT, as proved in the the 


HLC paper referenced above. 


To achieve this while not giving up the objectives on the 
small size of e, you can use NTP or a similar service to 
keep PT close to RT. 


In the end, the algorithm is simple, and perhaps even ob- 
vious in retrospect. The best things are. Simplicity is a 
core requirement for scalability and truly bug-free code. 
We hope these properties result in people using this 
technique. 
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Who's lying? 
by Jason Cohen on February 26, 2022 
A lesson all pilots know: How you must use multiple dials, 


employing different sources of energy, to report identical 
data, because some of it is always lying. 


ane ANDERSON 


“I know it’s wrong, I’m just waiting for the autocorrect.” 


We had been flying for four hours, but both gas gauges 
still read “full.” I didn’t need a pilot’s license to know that 
couldn’t be right, nor to feel the rush of adrenaline in my 
gums at the thought of the engine sputtering to an eerie 
quiet death, propeller blades windmilling as we scream 
“mayday mayday” and “set it down over there” like in the 
movies, hopefully including the part where the heroes 
confidently stride away while the wreckage ignites in a 
fireball, such a banal event in their exhilarating life that 
they can’t even be bothered to glance back at the 


carnage. 


“Umm, this can’t be right” I said to Gerry, the real pilot. 
“Yeah,” he said, “the needles get stuck to the glass.” He 
flicked the glass. The needle didn’t move. “So... do we 
have enough gas?” “Yeah, we have another hour left, I 


stuck the tanks before we left.” 


“Sticking” means plumbing a wooden dowl through top 
of the wing, into the gas tank, judging the gas level by 


the height of the resulting wetness. Sometimes the sim- 


plest technology is best. Wooden sticks don’t run out of 
batteries or make you wait forty-seven minutes for a se- 


curity update. 


Trust, but verify.” 
—Ronald Regan, repeating a Russian 


rhyming proverb: Jlogepati, Ho npoeepalt 


It’s not even good enough to just have “two of 
everything.” If two things both rely on electricity, and the 
electricity goes out, you lose both. There were two gas 
gauges; both failed for the same reason. It needs to be 
differently-implemented as well, like a stick versus a 
gauge. For example, there’s a normal magnetic ball com- 
pass floating in liquid that will work even if other power 
sources fail, but it’s hard to read as it bounces around 
from the vibration so it’s good to have another one that 
runs off suction—air pressure differential between the 
outside and inside of the cabin—which is stable even 


when you're turning the plane in turbulence. 


Gerry would say: “Who’s lying?” Usually your instru- 
ments are correct, but sometimes one is lying. Maybe the 
suction system isn’t working, so you double-checking suc- 
tion-based dials with the electric-based dials. You stick 


the tank, in case the gas gauge is lying. 


The same lesson applies to our daily life of data and met- 
rics. You think you understand what each number means, 
and usually you’re correct. But sometimes you’re running 
out of gas and don’t realize it. I’ve seen this happen for 
all sorts of reasons: The spreadsheet had a subtle bug in 
a formula, the analytics JavasScript code was accidental- 


ly left off one page, the survey email didn’t get sent to all 
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the customers in the cohort, the database query 
did/didn’t filter something important, a nightly update 


script hasn’t been running for three months. 


A good way to check for bad data is to replicate the air- 
plane dashboard method of deriving the same informa- 
tion in two different ways. Revenue from your billing sys- 
tem compared to cash flows from your bank statements. 
(Once I discovered our credit card processor was delay- 
ing our cash receipts.) Number of active customers from 
Stripe, and from your User Portal. (Because sometimes a 
cancellation in one system fails to cancel in another sys- 
tem.) Web traffic from Google Analytics but also another 
analytics system, or your raw web logs. (If you use five 
web analytics tools, they'll all give you different num- 
bers; this could be due to differences in definitions of 


things like “visit” and “session,” but is that truly all it is?) 


Besides paranoia, I’ve found another advantage in com- 
puting the same data twice: a better understanding of the 
forces behind that data, and therefore better analysis of 
how the company operates and how the market is chang- 
ing. Consider web traffic. There are analytics that tell you 
where traffic originates (imperfectly, especially with the 


latest browsers and extensions intentionally obscuring or 
blocking data), data about your advertising click- 
throughs, your own raw web server logs, and broad in- 
dustry data (e.g. Google Trends on how search-traffic for 
your keywords is changing). They all tell a different story. 
None has the full picture; all are biased. But taken to- 
gether, your picture of the world is more complete, and 
biases might be cancelled through averaging or by paying 
heed only to the clearest, most consistent trends. If four 
different sources agree that a trend is happening, then 


it’s definitely happening. 


If a metric is important enough to watch it every day, and 
to act if its behavior deviates from expectation, then it’s 
important enough to be double-checked. Both for accura- 


cy, and for completeness of comprehension. 


If your dashboard isn’t redundant, you’ll never know... 


who’s lying? 
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The Elephant in the room: The myth of exponential hypergrowth 


by Jason Cohen on March 5, 2022 


Even Facebook and Slack did not grow “exponentially,” as 
frequently described. Here is the correct model that you can 
use to understand and affect growth. 


A startup is growing fast, the journalists marveling at its 


“meteoric rise.” But don’t meteors fall? 


Inevitably it is breathlessly inducted into the class of “hy- 
pergrowth” companies that are “growing exponentially.” 
Especially when the product is “viral.” After all, if every 
person brings three friends, and each of those brings an- 


other three, is that not exponential? 


But “exponential” is an incorrect characterization, as 
we'll see in real-world data, even for hypergrowth, “viral” 


companies like Facebook and Slack. 


This article suggests an alternate model for how fast- 
growing companies actually grow. Understanding the 
model is useful not only for predicting growth, but be- 
cause understanding the foundational drivers of growth 
allows us to take smarter actions to create growth in our 


own companies. 


Dispelling “exponential” 


To evaluate whether hypergrowth is properly described 
as “exponential,” let’s recall what that word means. 
Here’s an exponential curve (like y = 2”), compared to a 


quadratic one (like y = x): 


Exponential vs quadratic curves 
15 


= Quadratic = Exponential 


In exponential growth, values grow by a multiple. For ex- 
ample: In year 1 you grow 10, in year 2 by 100, in year 3 
by 1000—each time the amount of growth is multiplied 
by ten. The compounding effect of multiplication causes 
the numbers to grow slowly initially, then skyrocket. The 
compounding effect gets journalists and VCs justifiably 


excited. 


Compound interest is the most 
powerful force in the universe.” 
—Albert Einstein 


In quadratic growth, values grow by a adding a constant 
amount more each time-interval, rather than multiplying 
a constant amount more each time-interval. In the same 
example, growing in year 1 by 10, then in year 2 by 20, 
in year 3 by 30: 
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+30 70 


+20 AO | 
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Successive values (in blue) are increasing more and more (in 
green). The green differences are increasing linearly: 10, 20, 30. 


Growth is still accelerating, so the blue curve slopes up- 


wards, but gently compared to exponential growth. 


With these patterns in mind, let’s examine real-world 
data, and see whether “exponential” is the right model. 


Facebook is the definition of hypergrowth—getting to 
$50B in revenue faster than any company in history. The 
product is “viral”—friends bring other friends—which 
theoretically leads to “exponential growth.” But Facebook 
didn’t grow exponentially in the number of monthly ac- 


tive users: 


Facebook: MAUs 
3,000 


2,000 


1,000 


MAUs (millions) 


0 
01-2005 01-2010 01-2015 01-2020 


Essentially linear for nearly twenty years, only exponential in 
the first four years. 


Slack was the fastest-growing enterprise software _com- 
pany ever, going from $0 to $10M ARR in their first 10 
months, and O to 10,000,000 active users in just five 
years. It’s also a “viral” product—organizations invite 
their members, who then create their own Slack-groups 
and invite others. So surely Slack has exponential 


growth? 


Daily Active Users ae slack 
12M 
10M 

8M 
6M 
4M 


2M 


0 
2014 2015 


2016 2017 2018 2019 


In September 2019, we exceeded 12 million daily active users 


Slack’s own data shows initial quadratic growth, followed by 
years of linear growth. 


If you compare Slack’s growth with Microsoft Team’s 
growth, do you still think Slacks growth is 


“exponential?” 
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Microsoft Teams vs. Slack 
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Dropbox was another “hypergrowth” company, achieving 
100,000,000 registered users five years after being 
founded in 2007, but it wasn’t exponential, neither in 


freemium users nor in revenue, early nor later in life. 


Number of registered Dropbox users from April 2011 to March 
2016 (in millions) 


600 
500 

500 
s 400 
= 400 
E 
£ 
rf 300 
@ 300 
3 
bh 200 
S$ 200 175 
E 
5 
= 100 

100 

50 
25 
> mm 
Apr ‘11 Jul'12 Nov ‘12 Jul'13 Sep '13 May ‘14 jun ‘15 Mar ‘16 
Source: Statista.com Years 


@ Number of registered Dropbox users 


Early in life, Dropbox registered users grows non-exponentially, 
nearly exactly 100M per year 


Dropbox paying users 


2015 2016 2017 2018 2019 2020 


Year 


Paying users (in millions) 
g 


Later in life, Dropbox revenue grows linearly, and slows down 


Trello grew fast too, getting to 10,000,000 registered 


users in three years. But not exponentially: 


Trello users 
16000000 


12000000 


8000000 


User count 


4000000 


12012 71/2012 1/1/2013 7/1/2013 1/1/2014 7/1/2014 1/1/2015 = 7/1/2015 1/1/2016 


Date 


Lyft grew in part due to “network effects” according to 
their $1, but this chart they presented shows that active 


rider growth isn’t exponential: 


Rides (in milions) 


Q6 Q2% O76 O46 QT Q27 O37) O47) QINB G28 O38 O48 


Hubspot’s revenue curve is astonishingly consistent, de- 
spite hitting multiple inflection points! in their business: 
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2 e.g. launching new business models like selling through agen- 
cies instead of only directly, launching new product lines like 
sales CRM on top of marketing automation tools, and scaling 
the number of customers and employees by 10x 


Revenue growth is strong 


$300 
$225 


$150 


Q3'14* Q1'21 
1) Revenue ($m) 


Analyzing this last example, we arrive at a new, non-ex- 


ponential model. 


Hypergrowth is quadratic 


The language we use can determine the thoughts we 
have. 


The Hubspot slide says “41% CAGR.” “CAGR” means an- 
nualized growth rate. They're saying that if you start 
with the first number on the slide, then from there plot 
growing 41% per each year, compounding each year 
upon the previous, for seven years, you would arrive at 
the last number on the slide. This is exactly the definition 
of “exponential”—multiplying by a number repeatedly. In 
general when you use “CAGR” or “percentage growth” as 
a metric, you are implicitly saying “This is an exponential 


process.” 


But Hubspot didn’t grow by 41% every year; in this time- 
frame, it started at 60% and ended around 30%: 


Hubspot YoY Growth Rate 


@ Revenue ($m) = YoY Growth 


$300 60% 
$200 40% 
$100 20% 
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Exponential curves have a constant year-over-year growth rate, 
therefore this is not exponential growth. 


If instead we examine growth in absolute dollars, rather 
than in percent, a pattern emerges. In the first set of four 
quarters on this report, they grew $17M. The next set 
grew $23M. Then $28M. Then $34M. Each year $5-7M 
more than the previous. This is the definition of a qua- 
dratic—adding an amount that increases by a constant 


amount each period, not multiplying. 


Charting these year-over-year revenue differences in ab- 
solute dollars rather than in percent, it’s clear that indeed 
the changes were almost completely linear for years, then 
suddenly changed in 20207 to a new (but still linear) 


rate: 


2 Coinciding with the launch of a new product: Hubspot CMS 
Hub. 


Hubspot YoY Revenue Change, in SM 
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It is therefore mathematically inevitable that plotting a 
quadratic curve (rather than exponential) on top of 


Hubspot’s revenue data will be a perfect fit: 


Hubspot Revenue: Best-fit parabola 
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When you said “best-fit,” you weren’t kidding! 


My thesis is that High-growth? companies grow qua- 


dratically, not exponentially. 


3 My guess is low-growth companies are similar, but data are 
more easily available for the runaway-growth companies who 
publicly flaunt their success. 


The consequence of this conclusion is important for oper- 
ators and analysis and investors. These are all people try- 
ing to understand—and possibly change—growth driv- 
ers. Getting the right language, and the right model, will 


lead to right analysis, and right action. 


Why marketing-driven products grow 
quadratically: A first-principles 
explanation 


Its not enough to draw best-fit lines on top of 
PowerPoint slides. We have to explain why this model 
makes sense, which in turn will create a better under- 


standing of the growth drivers in our own companies. 


We've been taking a macro view of growth, looking at 
multi-year trajectories. Now we'll peer into the micro- 
scope instead of the telescope, and consider how growth 


arises from a single marketing campaign. 


The life of a marketing campaign 
In my experience, marketing campaigns follow this 


pattern: 

Idealized shape of a marketing campaign 
125 
100 

75 


50 


Sales / Week 


25 


Weeks 


At the foot of the curve, we’ve launched a new campaign, 
but it’s ineffective; we haven't figured out the best design 
and messaging and calls-to-action for this new medium 
and audience. Sometimes we never figure it out, and 


abandon the effort* 


4 It’s hard to distinguish (a) our failure to build effective copy 
and conversion funnels from (b) channels that are fundamen- 
tally a bad fit for our market or product. This uncertainty, to- 
gether with the rapid evolution of digital marketing, suggests 
that we should retry campaigns in previously-failed channels 
every few years. 


But in the case that we unlock the secret of efficacy, the 
campaign rapidly reaches a natural level of contribution; 
in this example, a number of “sales per week.” The spe- 
cific level depends on many things: ad inventory, our 
budget, audience-receptivity, and the consonance be- 
tween the audience and our target market. 


Next we enter the optimization phase. We A/B Test our 
way to incrementally better results. Also we enjoy the re- 
sult of multiple exposures—most people need to see the 


ad more than once before they act. 


Finally we enter a phase of decline. There are various 


causes, all instructive: 
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e The audience saturates. Everyone in the channel 
has seen the ad more times than is required to act; 
it’s now falling on deaf ears. Even if the audience is 
growing, the number of new people is small 
compared to the number of people that were new-to- 
us when we began the campaign. 

¢ The channel declines. A media site that was popular 
loses readers through over-monetization. An event 
that was well-attended loses favor. A newsletter that 
was frequent and insightful becomes less frequent or 
other writers take over. A podcast moves to a closed 
platform and loses many listeners. 

e The auction becomes uneconomical. For auction- 
based systems like Google and Facebook 
advertisements, or other zero-sum programs like 
affiliates or limited-inventory spots on newsletters or 
podcasts, the winner is the one who will pay the 
most. What is cost-effective for one bidder will be 
laughably overpriced for another, due to better 
conversion rates, higher revenue per customer, 
higher profitability per customer, or due to 
categorization as a “loss leader” or other way of 


ascribing value beyond immediate pay-back. 


This curve leads to actionable ideas for managing mar- 
keting (given at the end of this article), but also forms 
my central thesis about how all sorts of growth works at 


companies. So I’m giving it a name: 


The Elephant Curve 
125 


Sales / Week 


How the idealized marketing campaign converts to 
growth 

The model above shows the number of sales per week the 
campaign contributes. To understand how this looks in 
terms of revenue growth, let us suppose a simple busi- 
ness model in which all sales are for a recurring revenue 
product generating $10 per month, with a 1% per month 


cancellation rate. How will revenue grow over time? 


Marketing campaign: Cumulative contribution to revenue 
$80 
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Growth initially accelerates as the campaign is solved, then 
grows roughly linearly as the campaign is optimized, and then 
starts sagging (although still growing!) as the campaign de- 
clines, and as the now-sizable customer base produces a non- 
trivial number of cancellations. 


The layer-cake of quadratic growth 

Marketing departments don’t stop at a single campaign. 
They add new ones. Some are bigger than others, some 
can be optimized more than others, some decline sooner 


than others, some decline more precipitously than others. 


So, let’s model that: A variety of Elephant Curves, with 
differing parameters, beginning at different times, stack- 
ing the revenue-contribution of each to arrive at overall 
revenue growth for the company. 
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Layered marketing campaigns 
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Layered campaigns create a “wavy quadratic.” 


Scan your eye across the top of this kaleidoscopic cake, 
and you trace a wavy quadratic. This makes sense mathe- 
matically, because each campaign is essentially linear af- 
ter it gets going, even if it sags during decline. “Adding 
more linear things over time” is the definition of a 


quadratic. 


The reason it’s “wavy,” is that when we unlock a new 
campaign we get a burst of growth. Do real-life revenue 
curves exhibit this waviness? Maybe so; here’s another 


slide from the Hubspot deck: 


The HubSpot Journey r9) 


Launch 


Hubspot didn’t just add new marketing channels, howev- 


er, but also layered on new geographies and new prod- 
ucts. Do those activities have the same effect as market- 


ing campaigns? 


Multiple product lines at marketing-driven 
companies: Still quadratic 

So far we’ve assumed a single product, driven by market- 
ing campaigns. High-growth companies who want to 
continue growing quickly after their first product reaches 


scale, must launch new products into new markets. 


Is the Elephant Curve also the shape of an entire product 
line? After all, products often have an initial slow-growth 
period (because only cutting-edge early adopters are ea- 
ger to pay to “be first” with bugs and missing features), 
followed by a faster expansion period, then reach some 
sort of natural ceiling, and possibly enter a period of de- 


cline (as the market evolves or competition overwhelms). 


Indeed, this is what we see with many products, especial- 
ly those that are marketing-driven, and without recur- 
ring-revenue. iPod sales, for example, are a perfect 
match: 


Quarterly iPod Sales Since 2002 


24000000 
18000000 
12000000 

6000000 


0 
Do coh oo co cit 6 62 ck cAN AD Ad cab 
Pa a a a SS 


It should therefore be unsurprising when we look at the 
overall revenue chart for Apple, and once again see qua- 
dratic growth on the top-line, admittedly with a special 
one-time bump for the unprecedented” success of the 
iPhone: 


5 It is rare for a second product to dramatically outpace the first; 
even juggernauts like Google, Amazon, and Facebook never 
achieved that. 
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Apple's Incredible 21st Century Growth 


Apple's revenue by operating segment since fiscal year 2001 
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2014, iPod sales are included in "Other product 
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Each product is in a different phase of its lifecycle: The 
iPod declined to zero, the iPad is still declining; Macs are 
teetering but essentially flat; iPhones and software ser- 


vices are still increasing. 


The quadratic explanation for “growth decay” 
It’s well-known that growth—as a percentage—naturally 
declines with scale, even when there’s nothing wrong 


with the company. 


This law of nature has been given a name: Growth Decay 
(or sometimes Growth Persistence). Because of the tradi- 
tional insistence of talking about growth as a percentage, 
the concept is articulated this way: If a company grew 
X% last year, it’s likely to grow a bit less than X% this 
year. With this formulation, the question becomes: How 


much less? 


The data give us the answer of 85%, although with 


R? = 0.51, this is a tendency but far from a law: 


Growth Decay in Recurring Revenue Companies 


Next Year CAGR 
> 


Current Year CAGR 


With our new appreciation that growth isn’t exponential, 
and therefore “percentage” might be the wrong way to 
characterize growth, we could ask what curve would best 
model the idea of Growth Decay? Specifically, let’s plot 
revenue for an initially-fast-growing company that is sub- 


ject to the principle of Growth Decay: 


ARR growth with theoretical growth decay 


Starting with $10M ARR at 120% YoY growth with 85% growth decay 
$3,000 
— $2,000 
7) 
= 
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The first leg of the curve is quadratic. While mathemati- 
cally not identical, the best-fit quadratic curve for this 


data has a staggering R? — 0.999. 


This is yet another signal that quadratic growth is the 


correct model. 


Beyond marketing campaigns: “Viral” 
and other forms of “exponential” growth 


But some products really do grow exponentially. In 


theory. 
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In theory, theory and practice are 
the same. In practice, they’re not. 


2 


—Benjamin Brewster 


Some products don’t grow proportionally with marketing 
and sales, but instead self-propel with a mechanism that 
theoretically ought be exponential. There are at least 


three ways for this to happen: 


1. Virality. When each user invites on average another 
a users, then each of those a new users bring in 
another a new users, so we end up with a? more. 
Then each of those brings in another a which yields 
a® . Then a‘ and so on; this is the definition of 
exponential growth. Biological viruses grow 
exponentially for a similar reason, justifying the 
label. 


Examples: social media, chat clients, peer-to-peer pay- 
ment platforms, massively-multi-player games, fantasy 


sports leagues 


2. Word-of-Mouth. All products have some word-of- 
mouth component, but here we’re referring to products 
that are primarily driven this way; this creates a growth 
process that is similar to viral. Typically the mechanism 
of “telling others” is built into the product, rather than 
bolted on by marketing or generated by goodwill. The 
difference between “word-of-mouth” and “viral,” is that 
viral products are unusable unless you invite others to be- 
come users (thus exponential growth is enforced) where- 
as word-of-mouth products encourage sharing. Thus chat 
clients are viral because without inviting others you can’t 
chat, whereas Wordle® was word-of-mouth, because you 
play the game alone, but are encouraged to share results 
on Twitter, which in turn brings in new users. 


6 Wordle exploded from 90 players in November 2021, to 
300,000 in December, to 2,000,000 in January, when it was 
bought by the New York Times. 


Examples: gamified products that generate significant 
sharing (self-improvement, game-results), consumer-to- 
consumer marketplaces where being a buyer plants the 
idea of becoming a seller (eBay, Airbnb, Uber); organiza- 
tions with a cause that creates on-going buzz (brazenly 
unique cultures, a passionate higher purpose, something 
people feel is linked to their personal identity). 


3. Hot Trend. Products that “everyone” (in some well-de- 
fined market) is going to buy. For smartphones, that 
might be half the population of the world. For internet 
search, that might be 100% of the online world. For 
backend management systems for large hospital chains, 
that could be 1000 potential customers. These products 
hit “tipping points” where “suddenly everyone buys it.” 
Even if, like internet search, the product has no explicitly 
viral nor word-of-mouth component—when you search 
on Google, you don’t “invite friends” to also search on 
Google—the ubiquity and inevitability of the trend leads 


to an explosion of users. 


Examples: word-processing, spreadsheets, broadband in- 
ternet, the smartphone, the shifts to cloud computing 
and online shopping, major media delivery platforms of 


radio, TV, DVD, and video streaming, 


Logistic growth: Nearly the right model for virality 

Products cannot grow forever, for the obvious reason that 
markets are finite. The Facebook virus spread to billions 
of people, but not infinite. Smartphones have been pur- 


chased by billions of people, but not infinite. 


Therefore, even if “exponential” is the correct model for 
the core growth mechanism of the product, it neverthe- 
less cannot continue growing exponentially because it 
runs out of market. Furthermore, markets tend to have 
so-called “low-hanging fruit’—customers who are more 
eager to buy—so although the virus spreads exponential- 
ly through these easy-pickings, it runs into the majority 


of people who will buy, but maybe later, maybe after 
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more of their friends or competitors are using it, maybe if 
it’s less expensive, maybe once it has more features, 
maybe once it supports integration with specific other 
software, and all manner of other excuses. The virus has 
more trouble infecting these high-strung fruits, so growth 


slows. 


This suggests a curve that starts exponentially, but then 
slows as it runs into the soft back-pressure of more de- 
manding customers, and finally flattens out completely 
as it runs into the hard limit of the size of the address- 
able market. 


Biologists have already done the work for us, because 
this is the correct model not just for viral products, but 
biological viruses infecting a population—akin to product 
types 1 and 2 above. Intriguingly, this is also the correct 
model’ for the diffusion of a gas across a membrane— 
akin to product type 3. The mathematical model for all of 


these processes is the logistic curve: 


7 The similarity is that in both cases you have a sudden demand 
that enters into a new space, but which slows and eventually 
stops as the new space becomes saturated. 


The shape of logistic = Exponential = Logistic 


10 


The logistic curve is exponential in the early days when it 
is far away from its natural limit. As the product (or gas 
or virus) gets to around 25% market penetration (or in- 
fections or saturation), the curve flattens into linear 


growth, in a tension between the exponential force of 


growth, countered by fewer and more demanding re- 
maining targets. Finally it levels out at what is called the 
“carrying capacity”—the fully-saturated market. 


The logistic curve is evident in the real world, in all three 


product types: 


80 


Twitter MAUs, in US 


Twitter is a type 1 “Viral” product that follows the logistic model 


Pinterest users 
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Pinterest is a type 1 “Viral” product that follows the logistic 
model 
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eBay users 
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eBay is a type 2 “Word-of-mouth” in the number of buyers, fol- 
lowing the logistic model (though also sagging towards the 
end, reminiscent of the Elephant Curve) 


eBay: Number of sellers: Logistic, then falling 
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eBay also follows the logistic model in the number of sellers 
(with even more pronounced sagging) 


Global smartphone sales 


Sales in million units 


@ Android @ ios Microsoft @ RIM @ Bada* @ Symbian* @ Others 


Smartphones are a type 3 “Hot Trend” that follows the logistic 
model all the way to saturation 


Percentage of web traffic from mobile devices 
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Smartphone usage, separate from smartphone sales, is also lo- 
gistic. Many pundits predicted this percentage would grow 
nearly without bound; in fact it saturated at 55%. 


Global Internet Users ® Users (millions) = YoY Change as % 
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The internet is a type 3 “Hot Trend” product with a near-exact 
logistic shape; at 66% global penetration, it hasn’t reached 
carrying-capacity, but it’s been in its linear mode for many 

years, and fell off the exponential path sooner than you might 

have expected 


Stacking logistic growth: The quadratic reappears 
Marketing-driven products demonstrated quadratic 
growth, especially once Elephant-shaped campaigns and 
products were stacked. How does this differ with logistic 
growth? 


As already pointed out, logistic growth is similar to the 
Elephant Curve. The “high growth” portion of a market- 
ing campaign might in fact be logistic; a product might 
extend that period into years rather than weeks, and the 
absolute magnitude of the result might be many times 


larger. 
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If this idea is correct, we ought to see viral-like products 
exhibit a similar curve to the iPod curve—i.e. a product 
with initially-exponential growth, then a flattening, per- 
haps with some small growth, then on a long-enough 
timeline, a decline. An Elephant with a more stretched- 


out trunk. 
The Facebook Messenger product appears to exhibit at 


least the first half of the logistic curve: 


Facebook Messenger Global Monthly Active Users 


In millions 
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Furthermore, this curve is actually a sum of US growth 
and outside-US growth. Looking only inside the US, 
Facebook Messenger is further along the curve, past the 
linear mid-section and already leveling out near some 


carrying-capacity: 


Facebook Messenger: MAUs, US (millions) 
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The same thing happens with Facebook DAUs and 
MAUs® DAUs in the United States and Canada are logisti- 
cal and have already topped-out at an apparent natural 


carrying-capacity of 185 million: 
8 Daily Active Users, Monthly Active Users 


Facebook's daily active users in the U.S. and Canada 
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Breaking out MAUs by all geographies reveals that top- 
line growth of users is an aggregate of some geographies 
essentially not growing at all (late in the curve), while 
others are still growing, albeit also linearly (middle of 
the curve): 


Facebook Monthly Active Users (MAUs) 


In Millions 
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The result of these individual effects of different prod- 
ucts, released at different times, in different geographies, 
each with a “marketing campaign” style growth curve, is 


that it adds up to linear growth: 
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Facebook: MAUs YoY change = MAUs (millions) = YoY % Change 
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Exponential growth for the first few years crashes down into 
linear growth for nearly twenty years, from large-scale logistic- 
shaped products and geographies 


Does this conform to the Elephant Curve? Is this really 
still essentially quadratic? The answer is clear when we 
plot the same data, this time measuring the year-over- 
year change in MAUs. Not as a percentage, but as 


numbers: 


Facebook: MAUs YoY change (numbers, not percent) 
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Facebook MAU growth is indeed an Elephant Curve: Logistic at 
first, then flat(ish), then starting to decline. 


Why do we keep seeing this pattern, even at the scale of 
Facebook, one of the most “viral” products of all time? 
Because mathematically, things that look like an Elephant 
Curve, even if the logistic “trunk” is elongated over time, 
are linear for nearly their entire lifetimes. Everywhere ex- 
cept the very beginning. Adding up linear things defini- 


tionally creates a quadratic. 


As a striking example of this claim—that multiple, vari- 
ous Elephant Curves result in quadratic growth in the 
real world—consider the detail behind the earlier chart 
of Global Internet Users over time, a type 3 product. 
Every country has grown logistically, at a variety of start- 
ing-times, diffusion rates, and carrying capacities, yet the 


aggregate is quadratic: 


Share of the population using the Internet 


Internet users are all who have used the Internet in the last 3 months 
100 


1995 1900 199 


08 2010 


Total number of people using the Interne 


Thirty years of varied logistic growth adds up to quadratic 
growth 


To be certain the graph at the bottom (which is the same 
data as the chart shown earlier) is specifically quadratic, 
we chart the absolute difference in online population 
year by year. In a quadratic, these differences should 
grow linearly, i.e. each year adding a constant amount 
more than the previous year added. Which is indeed 
what we find, as precisely as we could expect from data 
in the messy real-world: 
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Absolute change in global Internet users 
Each year 14M/year faster than the year before (R‘2 = 0.89) 
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Bringing it back down to the scale of a single company, 
consider Netflix, another type 3 product. While their 
overall growth accelerates, under the hood we can see 
the US is was already in a phase of slow-growth by 2014, 
with outside-US is taking up the slack through 2019: 


Netflix Total Paid Subscribers and Free Trial Users 


NETFLIX i 


Source: Company filings, 2019 
A quadratic top-line, created by two roughly-linear geographies 


If we chart the changes in subscribers, rather than totals, 
it’s even more clear that growth in the US has been in the 
declining phase of the Elephant Curve for a while, with 


outside-US is growing linearly: 


Netflix Suffers Subscriber Setback 


Netflix's paid net subscriber additions per quarter 
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And then, fast-forwarding to 2023, looking at total, glob- 
al subscribers, we see that growth slowed outside of the 
US as well, and the familiar Elephant Curve returns in its 


entirety: 


Netflix global subscriber count 
$250 


$200 
$150 


$100 


Subscribers (millions) 


$50 


$0 


Logistic growth with a varying carrying capacity: 

Start with market-share 

Suppose you're Facebook, and you've saturated many 
markets. You might be at carrying-capacity for those mar- 
kets, but more people are still coming online. The mar- 
kets are growing, so your carrying-capacity is growing, so 


you should still be able to grow too. 


Indeed, recalling the charts above, Facebook’s current 
MAU growth rate, and that of global Internet users, both 
are currently hovering around 7% per year. Which isn’t a 


coincidence. 
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Let’s plot Facebook’s MAUs as a percentage of people on- 


line—their market share. 


Facebook: MAUs as % of global internet users 
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The Elephant Curve strikes again 


Finally we have the complete answer to why Facebook’s 
growth appears so “linear,” when the theory expects an 
Elephant Curve. When you examine growth relative to 
market size it is an Elephant, complete with logistic 
trunk, optimized back, and declining rump (even despite 
a COVID bump). 


This is why at-scale companies are willing to spend bil- 
lions of dollars increasing the size of the market—it’s one 
of the few ways to create growth other than raising 
prices. So Google spent billions on Loon—a subsidized 
service to bring low-cost internet to remote areas of the 
world. Its problem-statement is the first text on its web- 
site: “Billions of people across the globe still don’t have 
reliable, affordable access to the internet.” Or, putting it 
another way, “Wifi balloons are a kooky idea but how 
else are we going to increase the carrying-capacity of the 


‘global internet user’ Elephant Curve?” 


Or Facebook with its “Free Basics” system that (in_their 
words) “Helps people discover the relevance and benefits 
of connectivity with free access to basic online services.” 
Except actually it’s only a few, hand-curated websites, all 
of which just happen to be western consumer products 
companies that are large Facebook advertisers, and the 
only available social network just happens to be 


Facebook. And there’s no email, so I hope you like 


Facebook Messenger. In other words, a digital colonialism 
whose purpose is to increase the carrying capacity of 
Facebook MAUs and the advertising that goes with it. 


Elephant Curves are more visible when we plot growth as 
market share, because this incorporates the idea that car- 
rying-capacity of the underlying market can itself be a 


moving target. 


Logistic growth with a varying unit revenue 

We've largely been analyzing users rather than revenue, 
and for good reason: The lifeblood of any product is peo- 
ple who use it, regardless how much money it can extract 


in the process. 


However, when we turn to revenue, we find that curves 
can become perkier. Facebook’s user growth might be lin- 
ear, but could it be that revenue is exponential? It’s cer- 


tainly not linear: 
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We already know Facebook’s user growth is linear, so the 


missing piece is Facebook’s revenue per user: 
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Facebook: Revenue per user 
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Logistic or Elephant yet again 


Perhaps by now we're not shocked to see the Elephant 
Curve once again. And we also know how the rest of the 
story goes: Because MAUs are Elephantine (which means 
mostly linear), and revenue-per-user is Elephantine 
(which means mostly linear), when you multiply them 
you get a quadratic, not an exponential, and that’s what 


we see in Facebook’s overall revenue growth. 


Actionable conclusions 


When we seek out the Elephant Curve in our marketing 
channels, product lines, geographies, and verticals, not 
just in its hopefully-explosive initial phase, but its phases 
of optimization and decline, we can proactively look for 
these phases, and take action. 


Model by component 
Our final discussion on the value of analyzing compo- 
nents of growth separate leads to a prescription for ana- 


lyzing growth. 


1. Estimate the growth curve for the entire market. 
Expect to be Elephantine (or simply logistic, in the 
case of trends that you can reasonably assume will 
not decline in the forecasted future, like global 
Internet use or smartphone use). 

2. Estimate the product market-share curve. Expect to 
be Elephantine, and don’t be so bold as to assume 
your product will never decline relative to the market 


—are you better at execution than Facebook? 


3. Estimate monetization, i.e. revenue per customer”. 


This curve might be Elephantine, but not necessarily. 
It is highly dependent on the product and market, on 
how distinct the product is competitively, on the 
budgets of the customers, and more. Facebook has a 
strong moat (network effect) and doesn’t charge end- 
users, so they (like Google) can raise prices 
consistently. A product in a commoditized market 
might never be able to raise prices, and thus must 
find growth in avenues like increasing usage, the 
introduction of companion products, expanding to 
other verticals or geographies, or by applying their 


technology to new markets. 


9 The definition of “customer” should match whatever activity is 
most highly correlated with growth; this is also what “market 
share” should mean. For normal products people pay for, this is 
simply “paying customers,” but for example in the case of 
Facebook, this is MAUs at least, perhaps even DAUs. 


You get better models by predicting each of these compo- 
nents separately, then multiplying for a final growth pre- 
diction. You’re also better able to track the model against 


reality, as more data becomes available. 


Besides this break-down, there are many operational 
ideas suggested by the results above, especially for man- 


aging marketing campaigns. 


This might be expanded in a future article, but for now, 
these probing questions might lead to better ideas on 


how to analyze and affect growth: 
Advice for Marketing teams 


¢ Is our AdWords campaign topped out? Are we fooling 
ourselves into thinking there’s more inventory to ac- 
cess? How much more optimization is there to be 
had, and how would know? Are we hitting a decline 
due to uneconomical auctions, and if so, what is our 
reaction? When should we start experimenting with 
new channels, rather than continue to flog the 
AdWords channel for results that don’t exist? 
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Is it ok to be less cost-effective if it means we can 
stave off decline? Should we be that “irrational bid- 
der” who bids “too much” because we’re wise enough 
to see value beyond immediate cash pay-back? If so, 
how do we quantify that value, so we know just how 


“irrational” to be? 


To hit our growth goals for the year, what would 
have to be true of the growth of existing campaigns? 
Which can be reasonably expected to grow, hold 
steady, or shrink, based on their phase? How many 
additional, successful campaigns do we need, and 
how soon? Since not all that we attempt will suc- 
ceed, how many do we need to start to yield the final 


quantity we need? 


Rather than stack up small, limited campaigns, is 
there something more substantial that could generate 
more total growth? A single, large new geography in- 
stead of many smaller ones? A single, substantial 
new sales mechanism (e.g. reselling) rather than 
more advertising? A different pricing model instead 
of an additional sales model? Even if it takes 10x the 
effort, and possible even if it takes 10x the time, it 


might have 10x the results. 


Or the reverse—do we pull funds when we smell de- 
cline, rather than spending our time and money 
fighting a losing battle, accepting a short-term hit on 
top-line growth in exchange for more efficient 
growth? Do we try to stack up many smaller, more 
efficient campaigns, generating growth as a bulk ef- 
fort? Each effort affects the top line only marginally, 
but conversely our growth is less sensitive to the de- 


cline of any one campaign. 


Advice for Product Managers 


It’s great to add a feature to an existing product, but 
significant additional growth comes from increasing 
carrying capacity or creating a new avenue of 
growth. Early on you should focus on winning mar- 


ket share in one space, creating the first Elephant 


Curve, but after the product matures, something 
more drastic is required: Wholly new products, or 


updates significant enough to address new markets. 


e It’s well-known that companies need to add addition- 
al products to continue fast growth after achieving 
scale. However the second product is highly unlikely 
to achieve same market share and monetary scale as 
the first, so there needs to be multiple, not just one !® 
This requires serious investment, parallel efforts, and 


the chutzpah to kill off the ideas that aren’t working. 


10 This is true at any scale—advertising is still 82% of Google’s 
revenue; of that 71% is advertising from search alone (i.e. ex- 
cluding YouTube and other properties). Apple revenue is 60% 
iPhone. Even at smaller scales: Basecamp (neé 37signals) built 
multiple products over nearly two decades but only their first 
was successful enough to be worth working on; the company 
divested itself of the rest and rebranded to be identical to that 
product. It is possible for second products to eclipse the first; 
the iPhone was of course not Apple’s first product; The Tesla 
model 3 outsells the earlier model X, And at my own company 
Smart Bear our second product ended up being 95% of sales, 
and we essentially did the same as 37signals and went to a sin- 
gle product model. 


e Because word-of-mouth-driven growth is so much 
more effective than marketing-driven growth (both 
in cost-per-customer and in that unlike direct 
advertising it grows automatically as the company 
grows), it is worth a great deal of time trying to 
figure out how to build that into the product, rather 


than relying only on the marketing team. 


Half my advertising is wasted. I 
just don’t know which half.” 


—John Wanamaker 


Mr. Wanamaker made his famous complaint more than a 
hundred years ago; even with modern analytics, today 
it’s worse. The quadratic growth model won’t solve that 
puzzle, but the better you understand the mechanisms of 


growth, the more it is under your control. 
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Extreme brainstorming questions to trigger new, better ideas 


by Jason Cohen on March 26, 2022 


We know, “no idea is a bad idea,” but brainstorming is often 
unsuccessful. These prompts actually work. They could 
even lead to a unique business model. 


“Boy, what a day, huh? 


How do you come up with fresh, transformative ideas? 


“Brainstorming” is hard—staring at a blank whiteboard, 
wondering whether someone could make a real-life “dark 
mode” whiteboard, then realizing that’s what a black- 


board is, only dustier. 


Or the modern version, seventeen pointers flying around 
a Miro board, zooming to 2000% to read that auto- 
scaled-down text in each standard-sized virtual sticky- 


note. 


We're blinded by our daily work: No forest, all trees. It’s 
too easy to glom onto an idea that’s been knocking 
around for year, its importance undeservedly increased 
by the familiarity of repetition. It’s hard to think past the 
last seven customer interviews or support tickets or sales 
calls. Those are great for generating tactical ideas that 
fuel our roadmaps, but they are tiny incrementalisms 


that cement us into tiny thinking. 


The following prompts jostle you out of tiny thinking. 
Each stretches some dimension of reality to an extreme. 
So extreme that it is nearly nonsense. But dramatically 
different perspectives can reveal distinctly new ideas. An 
idea that would be a 60% solution in an extreme hypo- 


thetical case, could be a 2x or even a 10x idea in reality. 


Sometimes the extreme is surprisingly appropriate. 
Unique business models emerge when at least one di- 
mension is so extreme that it defies critics and competi- 
tors to even conceive of its possibility’ A fantastic idea 
fulfilling the right extreme can be a company’s entire 


strategy, unlocking a long-term competitive moat. 


1 All of these were considered impossible barriers or business 
models, until a company did it anyway and won because of it: 
Zappos’ free shoe returns 364 days after purchase, Robinhood’s 
$0 trading fees, Amazon’s free delivery with Prime, Netflix’s 
mail-order DVD rental, open-source software companies charg- 
ing for something that’s 100% free and 0% secret intellectual 
property (now hundreds of billions of dollars in combined pub- 
lic-company value), AirBnB, Uber, and Lyft trusting that 
strangers will trust strangers in the intimate spaces of cars and 
homes. 


It’s worth a try. 


10x prices 


If you were forced to increase your prices by 10x, what 
would you have to do to justify it? 


What sort of brand looks and feels like something that 
expensive? What positioning would you take? How 
would the design of both the website and the product 


need to change? 


What subset of your market would you have to target? 
Do they have different problems that need solving, or dif- 
ferent needs? Would they consider the high price a posi- 
tive, because it fulfills a need to be seen as someone who 
is successful, or because they feel “buying from the best” 


lowers the risk of their decision? What would you need 
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to do for them to be able to display that badge? What 
sort of relationship would you want to cultivate with 


each customer? How would your business model change? 


What expensive services might you need to supply? 
Human support? Infrastructure? Is it possible for those 
costs to be “only” 2x or 3x more than today, so that the 


net impact is massive profitability? 


Often early startups charge too little, and established com- 
panies struggle to charge more to existing customers. 
Thinking about what could justify a massive hike, you 
might be able to do some of those things and justify at least 
2x. 


No customers 


If all our customers vanished, and we had to earn our 


growth and brand from scratch, what would we do? 


How would we distinguish ourselves with a unique com- 
pelling message, as a fresh competitor in our market? 
What would we put on our home page? Would we use a 
different pricing model to compete better? Would our 


brand need to change to align with the message? 


Would we throw out some features that take a lot of 
work but not many people actually use (since there’s no 
pesky customers hanging on to the expectation of the old 
features)? Would we build some new feature that would 
make us more competitive? Would we change our in- 
frastructure or architecture or UX dramatically because of 
what we now know, since we're not ladened with existing 
customers? 


Often we don’t make important changes because we’ve got- 
ten complacent with our marketing, or we never got around 
to having a truly compelling unique positioning statement, 
or we don’t want to incur the penalty of big changes on our 
customer base, even if it means we’re doing the wrong thing 
today from a competitive standpoint, and the wrong thing 


for the future. We don’t want to make even 5% of our cus- 


tomers mad, even if it would be better for the other 95%. 
There will be 10x more customers in the future than there 


are now, but only if we build for them, today. 


No tech support 


"SO, aS you Can see, customer satisfaction is up 
considerably since phasing out the complaint forms." 


If you were never allowed to provide tech support, in any 


form, what would have to change? 


How would on-boarding need to be improved, to the 
point where customers would self-serve and be happy do- 
ing it? This might be the hardest step, because the cus- 
tomer is least-familiar with your product, and least-moti- 
vated to power through barriers to their success. And set- 
up stuff often has to be done only once, so you might not 
learn how to improve those things from existing 


customers. 


Where would the product need provide the user with 
more control, since they can’t ask Support to do it for 
them? What information requires better visibility, since 
they can’t ask Support to look it up for them? What ac- 
tions would need to be become more intuitive, because 
they can’t ask Support how to use functionality that they 


know “is in here somewhere?” 


Self-service isn’t just better for your costs and scaling, it 
also makes customers happier and more loyal. Therefore, 
dramatically improving self-service is often profitable for 


both the top- and bottom-line. 
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Maximize fun 


What would be the most fun thing to build? 


It has to be something that makes our product better, not 
a random technology or unrelated market. Aside from 
that constraint, it can be anything—building a feature 
you just think would be cool for customers to use, devel- 
oping a technology that would be fun to work on (that 
happens to deliver a feature customers want), refactoring 
infrastructure or architecture using some interesting 
modern technology that would also improve something 
like cost-efficiency, scalability, testability, or 


maintainability. 


What if you held a one-week hackathon and actually 
tried to build one of those things? 


When we work on things that are fun, we work better and 
harder, yet are happy to do it. Therefore, biasing towards 
fun is also biasing towards productivity and long-term ful- 
fillment, which is practical and logical, not merely indul- 
gent as it first appears. Ultimately, work still needs to ad- 
vance the strategy while improving something substantial 
for the customer or the team, but typical prioritization un- 


der-appreciates the power of fun. 


Complete rip-off 


If our biggest competitor copied every feature we have, 


how would we still win? 


Is the answer inside our product, in something other than 
the utility of the features? What is that, and how could 
we make more of that, so we’re differentiated even in the 
face of a copy-cat? Is it ease of use, ease of sharing, plea- 
sure of great design? Is it building the next unique fea- 
ture so quickly that no competitor can catch up (since in 
this hypothetical they’re just copying us rather than being 
insightful on their own), and if so, what is the killer next 
feature that would leap-frog us from a customer’s 
perspective? 


If the answer is outside of our product, where is it? A 
higher purpose or brand-promise that our customers buy 
into, aside from the product? Is that because we are good 
at communicating how we make the world better, or be- 
cause our customers are “rooting for David against 
Goliath,” or how our culture is special, or because our 
brand is distinctive (even if just “it’s the best” or “it’s the 
biggest”), or because our customers trust us, maybe be- 
cause of our customer service, or because of how we han- 
dle situations that go poorly, or because we “give back” in 
forms like open-source or community-building or signifi- 


cant philanthropy? 


All your good ideas will be copied; it’s just a question of 
when. Competing only on features results in bullet-point 
battles; this is the weakest way to win sales. Creating big- 
ger and more emotional distinction is a powerful way to 
win, and breaks us of the habit of believing that incremen- 
tal product updates will dramatically increase differentia- 


tion, or growth. 


No time 


What if we are forced to ship a full, completed MVP (or 
actually, SLC) new feature, in just two weeks, that would 


delight and surprise some fraction of our customers. 


To achieve great things, two 
things are needed: a plan, and not 
quite enough time.” 


—Leonard Bernstein 


Every bit of complexity has to be stripped out—we can 
add it back later. Scope has to be reduced to a minimum, 
which reduces how many people can use it on day one, 


but increases how much we'll learn on day one. 


Can we stub out an API, implementing it manually for 
now? Can we hook up some 3rd-party system, imple- 


menting something more efficient later? Can we skip 
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documentation and internal support training and perfect 
error handling, not forever, but just long enough to see 


how people like this new functionality? 


A 1000-page book can be summarized in a 40-page Cliff’s 
Notes. The key lessons of Cliff's Notes can be summa- 
rized in a 1-page blog post. The main theme and one 
unique perspective of the book can be summarized in one 
sentence. It can always be smaller, if we trade off detail, 


or precision, or completeness. 


Hackathons prove that we can code really cool things in a 
short time, when we want to, and if we make certain trade- 
offs. It’s not fully finished, not polished, not ready to scale, 
but it’s something substantial and far more “agile” than 
teams normally are. If you feel like you've lapsed into a wa- 
terfall with two-week report-outs, this might be the shake- 


up everyone needs. 


Flipped business model 


“I was thinking — instead of an allowance, how about 
more of a subscription model.” 


What if you were forced to charge customers in a com- 
pletely different manner? The product isn’t free, but 
somehow you have to justify value (and costs) different- 
ly. If you’re usage-based, you’re forced to charge a flat 
monthly rate; if monthly with tiers, you’re forced to mea- 
sure and charge daily with some formula. 


How would you have to change how you position the val- 
ue you deliver? Would you have different kinds of cus- 
tomers due to prices effectively being much higher or 
lower, and how would that affect brand, messaging, mar- 


keting, or sales? 


How would costs have to change, so that the vast majori- 
ty of customers were still profitable? Would software ar- 
chitectures need to change? Would marketing or sales 
structures change? Is tech support suddenly much more 


or much less possible? 


Are there features which you’d need to build (e.g. so cus- 
tomers could control their costs), or features you’d now 
be able to supply (e.g. because customers would now see 
the utility in it, or would be happy to pay more because 
of it)? Would you create features which caused higher 
prices, but in concert with additional value, so that cus- 


tomers would be happy to grow with you? 


Business models force you into certain frames of thinking 
about how to monetize value that you create, and how 
costs need to scale with sales and infrastructure. The gener- 
al idea for any company is to generate far more value than 
it charges for, so customers are genuinely better off in the 
exchange, and are happy to be long-term customers. 
Shaking up the business model shakes up the value/cost 
equation; sometimes a different business model is actually 


better for everyone. 


No website 


If you were not allowed to have a website, how would 
you still grow your business? 


Could you grow by word-of-mouth? Does the product 
help people do that proactively? 


Could you create advocacy, as in channel partners or am- 
bassadors? Could your social media be so good, someone 
would sign up for the product without needing to know 


more? 
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Could your product be marketing? Could you cheat, 
where your “website” is inside the product, where poten- 
tial customers are caught up learning more and trying 
things in the moment, so that they become users before 


they realize what’s happening? 


Your advocates are already using your product, yet so often 
we leave “getting more customers” to the marketing depart- 
ment, rather than realizing the product is already a plat- 
form for growth. Truly viral products know this already 
(e.g. you can’t use a chat service unless you invite other 
people to the chat service), but non-viral products can still 
create growth machines from inside the product. What if 
“in-product marketing” is more impactful on growth than 


any useful feature? 


No meetings 


What if you made your most introverted teammates’ 
dreams come true: No more synchronous meetings, ever 


again? 


What if you had to on-board new employees without 
meetings? Or operate the software, deploy the software, 
find where things are located? What systems would need 
to exist, and what materials would need to go in there? 
What format would allow you to update that easily? 


What if you had to make decisions without meetings— 
the process as well as the result of that decision? 
Deciding what to build, prioritizing big or small things, 
brainstorming new things? Is it possible that brain- 
stormed ideas could be even better without meetings (be- 
cause people aren’t being influenced by other people, 
ahead of having their own thoughts) or that decisions 
could be even stronger without meetings (because people 
have the time and space to think, to research, to try 
things out, thus more completely exploring the solution 


space?). 


How would you create social ties and inside jokes and 
1:1 as well as group relationships, if you’re never in the 


same space at the same time? 


Meetings are still the best way to accomplish certain things: 
Social bonding, deep discussions where ideas rapidly rico- 
chet off each other, decisions where it’s important to “look 
in everyone’s eyes” to get the final agreement, and more. 
But one of the most common complaints in any company is 
“too many meetings,” or at least, “too many useless/bad/ 
wasteful/inefficient/boring meetings.” Besides the usual ad- 
monitions about meeting hygiene, a bigger question is 
whether your systems and processes can not only prevent 
them, but increase utility beyond that. For example, how 
much faster could a new team member get up-to-speed if 
they could read “the why” behind the last 20 major 


decisions? 


No customer contact 


If we could never talk to our customers again, how 
would we figure out what to build? 


Could you measure their behavior so well that you could 
quickly measure whether any change was positive or neg- 
ative, so at least you could iterate your way to a better 
product? Are there clear signs of value or happiness that 
you could use like marketers use conversion with A/B 


testing? 


Could you be even faster at testing new ideas? Could a 
feature start out as just a button that says “coming soon” 
or asks “why did you want this” or “what did you expect 
to happen” or “get emailed when we make this feature,” 
so you could directly measure what people are likely to 


use, before you invest in building it? 


What could you analyze online, in what customers or 
competitors or observers are saying, that would inform 
your strategy or even product roadmap? Could you do 
better than just copying what others are doing? Could 
you analyze what people are saying about us or competi- 
tors online, understanding that the loudest voices aren’t a 
random sample of the population, but still might have 
something useful to tell us, even when they’re not being 


constructive about it? 
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Sometimes you interview customers and come away with 
the wrong feature ideas anyway. Sometimes you ask them 
whether they’d use something, and they say yes, and they 
even meant it, but after spending five months building it, 
they don’t actually use it. Stretching your ability to get em- 
pirical signal helps you avoid those issues, and might even 
create breakthrough ideas, or avoid ideas that seem great, 


but aren't. 


Cost is no object 


What if it didn’t matter how unprofitable you were? 


Maybe prices stay the same but you can spend 10x or 
100x more, even if that’s wildly unprofitable. Or maybe 
the product is free, too. How would you spend the mon- 
ey? How much value could you deliver, if you thought of 
the product as an exercise in philanthropy? 


It could be infrastructure or software, but what about the 
personal touch? What would a dedicated, personal 
concierge do? If software could do just 30% of that job, 
but at 1/100th of the cost, that could be a good feature. 


It can be surprisingly difficult to convert money to real cus- 
tomer or business value, even when money is free. If there is 
a way, it might lead to a new pricing tier, leveraging that 


idea but adding back a sustainable business model. 


Sociopathic CEO 


; 


“So here’s how the restructuring is going to work. 
I’m going to be Tom. Tom, you’re Anne now. 
Anne will be the copier, and, going forward, 

the copier is in charge. Questions?” 


What if you could change anything, regardless of what 


anyone thinks or feels? 


Terminating an entire product line would shake the orga- 
nization, and possibly incur layoffs in multiple depart- 
ments. But what if it’s the best thing to do, despite that? 
What if you allowed yourself to explore what that would 
be like? What if it turns out you can make that pivot by 
reorganization rather than layoffs, and now company is 


executing a much better strategy? 


Pivoting the entire company could break the organiza- 
tion, destroy trust in leadership, cause people to leave, 
and piss off investors. But what if that’s exactly the trans- 
formation required to fix the strategy? Is there anything 


that might actually be worth such a move? 


You never want to contemplate catastrophic actions like 
layoffs, whole-company pivots, terminating an entire prod- 
uct line, and for good reason—the consequences are dire, 
arguably even inhumane. At the same time, that aversion 
prevents you from thinking certain thoughts. You should at 
least be able to explore the thoughts. The very best ideas 


should not be avoided only because they are hard. 
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Mortal wound 


What externality has the potential to kill the entire 


company? 


Is there a company X, which released product Y, at price 
Z, that would reduce your new sales to a trickle, and 
would mean half your customers leave within a year? 
Can you think of a security breech so significant, most of 
the customer base leaves within a year? How could the 
economy change, in which no one would buy your 


software? 


Now, is there something you could do to mitigate that ef- 
fect? Or something that pivots the company such that it 


is no longer a threat? 


I ran this exercise at WP Engine and we came up with 35 
existential threats; it’s not wise to worry about all of them, 
and try to negate all of them. Sometimes, however, you find 
an idea which is wise to do anyway, that also shores up 
your defenses. Or, one threat sticks out as being so likely 


that it warrants acting ahead of time. 


Philanthropist 


What if our only goal were to create the most good in the 


world, personally for our customers? 


Ignore how to make money; assume that works itself out. 
But still talking about our customers (don’t argue about 
which world-wide causes are “most important”). How 
could we make their lives better, not because our product 
has greater utility, but because our product is doing 
something important. How could we help them achieve 


their ultimate goals rather than just “use features?” 


If our customers are small businesses, could we help 
them grow and thrive? If our customers are writers, 
could we help them improve their craft? If our customers 
are performers, could we get them more attention? If our 
customers are enormous companies, could we help them 


use their greater scale and wealth to create more good in 


the world? If our customers come from all walks of life, 
could we increase the access and equity in their reach 


and power? 


There’s already strong evidence that having a mission big- 
ger than yourself more than metrics, creates more—and 
surveys of Gen Z show that they care about whether a com- 
pany does good in the world just as much as they care 
about compensation. So, even aside from the ethical rea- 
sons, there are “self-interested capitalist” reasons to create 


more good, not just more MRR. 


Only one thing this year 


What if you could only ship one thing this year? 


What single initiative would make the most difference? 
What’s so impactful that it would actually be OK if we 
shipped nothing else? You get the entire year, so there’s 
time for a release, and fixes, and enhancements, but it 


has to be only one topic. 


What would generate the most revenue? What would be 
so differentiated in the market, and so desirable, that 
customers still buy and stay even with your lack of any 
other substantial product changes? What would be so ex- 
citing that customers would stay even if other things are 


missing or have on-going bugs? 


When you look back over years of a business, often the 
whole trajectory comes down to 1-2 big decisions per year. 
A critical product launch, a key decision to enter a market 
for expansion or exit one market to focus on a healthier 
one, a key hire, a competitive insight. It’s difficult to know 
in the moment which will be seminal, but thinking this way 
forces you to think of only the absolutely most impactful 
ideas, which are probably the ones you should be focused 


on regardless. 
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The dangerous man is the one 
who has only one idea, because 
then he'll fight and die for it. The 
way real science goes is that you 
come up with lots of ideas, and 
most of them will be wrong.” 


—Francis Crick 
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by Jason Cohen on April 9, 2022 


This admonition recurs in myriad books, frameworks, and 
topics, across decades of time. When something is so 
consistent, it must be wisdom. 


“Turns out my imposter syndrome was pretty much 
right on.” 


A primary blocker of progress, from our personal lives to 


our corporate strategies, is a 
Failure to face the truth 


“The truth hurts.” Yeah, so we avoid it. The truth is hard 
to find if we’re not looking for it, and we’re not. Denial 
(when intentional) and rationalization (when uninten- 
tional) is our normal operating mode. Because we don’t 
realize this, it’s an immense, invisible barrier. Whether 
because we don’t like to admit we’re wrong (even to our- 
selves, in secret), or because it will be annoying or 
painful or career-altering for the truth to be said aloud, 
we avoid the truth. 


Once you start seeing the pattern of “failure to face the 
truth,” you see it everywhere. In almost every meeting, 
someone is thinking something and not saying it, even 
though one of the best uses of a meeting is to unveil and 


discuss insights. In every strategy discussion, there’s a 


monster in the room no one will name, even though the 
point of strategy is to identify and then construct a battle 
plan against the monsters. Written plans and strategies 
are all optimism and confirmatory data, rather than crys- 
talizing the scary challenges so that we can attack them 


together. 


Those who break through this barrier are rewarded, as 
many famous books and frameworks point out. It’s fun to 
see this in action, because the examples are individually 
interesting, and because a totality of instances illumi- 


nates the pattern. 


Radical Candor 

“When you say ‘um’ every third word, it makes you 
sound stupid.” Not the feedback Kim Scott was hoping 
for after what she thought was a successful presentation 
to executives at Google. And it came from Sheryl 


Sandberg of all people. (Before she joined Facebook). 


Now Scott knew, that Sandberg knew, that Scott wasn’t 
stupid. Therefore, it was obvious to her that this feedback 
was meant to help, not “obnoxious aggression” as Scott 
would later name the same straightforwardness when the 
motives are to belittle and hurt rather than to coach. And 
so Scott was grateful for being made to “face the truth.” 


In her words: 


Why had nobody told me for 15 years? It was like I'd 
been walking through my whole career with a hunk of 
spinach between my teeth and nobody had had the 
courtesy to tell me it was there. 


The truth is a courtesy; conversely, failure to face the 
truth is at least impolite, and at worst “ruinous” in Scott’s 
terminology, as when a manager withholds necessary 
feedback for fear of hurt feelings or being seen as 
unkind. 


1 of 4 


Failure to face the truth 


Scott is now famous for promulgating this philosophy of 
direct, honest, empathetic facing-of-the-truth in her 
book, Radical Candor. It starts with a foundation of per- 
sonal trust—trust that the feedback-giver genuinely cares 
for the well-being of the feedback-receiver, and that it is 
given in the spirit of “facing the truth” and not in the 
spirit of domination or manipulation or other ill-intent 
that people in positions of power might visit upon those 


subject to that power. 


We’re thankful when a coach admonishes us and shows 
us a better way; a coach that lied to you, even by omis- 
sion, telling you everything is fine when it isn’t, would be 


a bad coach. 


Five Dysfunctions of a Team 

Possibly the most famous book on diagnosing problems 
in team dynamics and building high-functioning teams, 
Five Dysfunctions by Patrick Lencioni starts with the ob- 
servation that great teams are built in layers, each requir- 
ing the next to be effective: 


LENCIONI: 5 (DIS)FUNCTIONS OF A TEAM 


Focus on personal/ego/own department. 


Not holding one another accountable. 
‘Accepting mediocracy, not taking 
action/initiative 


Not involving people in decision making, 
saying 'yes' doing 'no' 


Artificial harmony, not discussing real issues, 
avoiding conflict, not speaking up/listening 


Invulnerability, not real, absence of trust 


Tay MORROW’S 
LEADERSHIP 


Exactly like Scott’s Radical Candor, it is critical that a 
team be capable of constructive conflict, “discussing the 
real issues,” facing the truth. That requires the team to 
first build trust, so that it is safe to have conflict. 


“Failure to face the truth,” together, in the open, prevents 
teams from being great. We must have enough trust in 
each other to face the truth, together. If someone can’t 
handle that trust, maybe they’re not right for the group 


after all. 


Good Strategy, Bad Strategy 

In one of the seminal books defining “strategy,” Richard 
Rumelt lists common hallmarks of bad strategy. Each one 
is so common, you'll shake your head and laugh (or cry, 


if you’re guilty of these sins): 


¢ Fluff. Fluff is a form of gibberish masquerading as 
strategic concepts or arguments. It uses [non-specific 
or] esoteric concepts to create the illusion of high-level 
thinking. 


¢ Failure to face the challenge. Bad strategy fails to rec- 
ognize or define the challenge. When you cannot de- 
fine the challenge, you cannot evaluate a strategy or 
improve it. 


¢ Mistaking goals for strategy. Many bad strategies are 
just statements of desire rather than plans for overcom- 
ing obstacles. 


¢ Bad strategic objectives. A strategic objective is set by 
a leader as a means to an end. Strategic objectives are 
“bad” when they fail to address critical issues or when 
they are impracticable. 


—Richard Rumelt, Good Strategy, Bad Strategy, p32 


His phrase “Failure to face the challenge” is my inspira- 
tion for “Failure to face the truth.” Strategy must identify 
and then address the most important and difficult facts- 
of-the-matter of the market, competitive space, cus- 


tomers, product, and team. 


It’s scary to say “Our market is shrinking,” but if it’s true, 
and you refuse to identify it, if you don’t write it down 
that crisply, if you don’t challenge the company to come 
up with alternatives for how to address it, if you don’t 
build a strategy that expressly attacks it head-on, then it 
will be fatal. Face the truth. 


Lazy language belies a deeper failure 

There are traces of “failure to face the truth” even in 
Rumelt’s other bullets. Fluffy language is a personal 
peeve of mine. In the benign case it is simple laziness— 
avoiding the work of crafting prose by filling the requisite 
space on the page with jargon and generic phrases. In the 
worst case, it belies the lack of having thoughts in the 
first place. 
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This problem is rampant in marketing and content-mar- 
keting, but sticking with the theme of strategy, it’s espe- 
cially common in “vision” or “mission” statements, theo- 
retically summarizing the company’s aspirations, but of- 
ten just a non-specific blob that also applies to any other 


company in the space, e.g. 


A leading provider of website development for busi- 
nesses of all sizes. 


Well, that was easy! Easy because it says almost nothing, 
and as a result, it’s wrong. Wrong because it doesn’t de- 
velop websites both for coffee farmers in Ethiopia and 
also for Tesla’s launch of their next vehicle (“all sizes”). 


Perhaps it’s mere laziness; after all, the only words that 
communicate anything about what it is, is “website de- 
velopment.” Or perhaps the company refuses to “face the 
truth” of what it really does, and who it really serves, 
where it’s really strong but also weak, afraid to say “no” 
to any potential customer. What if, instead: 


We are boutique artisans chosen by discerning busi- 
nesses who demand the highest-quality, completely 
unique web experiences. 


Sounds like a small company (“boutique”) who neverthe- 
less charges a lot (also “boutique”), but delivers amazing 
work (“discerning” and “unique”). By saying “no” to peo- 
ple who don’t want that, you get to say “yes” to interest- 
ing projects at profitable rates, even beating out larger 
competitors who you can argue are just “factories that 
churn out the same website with different colors.” But 


only if you face the truth. 


Confront the brutal facts 
In Good to Great, one of the most-cited books on the for- 


mula (if such a thing exists) for successful businesses, 


Jim Collins frequently returns to the story of A&P and 
Kroger—two companies alike in dignity in fair 1960s 
America, where we lay our scene—yet Kroger catapulted 
past A&P because, in Collins’s words, only Kroger was 


willing to “confront the brutal facts.” 


KROGER, A&P, AND THE MARKET 
Cumulative Value of $1 Invested, 
1959 - 1973 


General Market: 
#4 $3.42 


$2 Kroger: $1.26 


7 A&P: $0.64 
1959 1966 1973 


Kroger and A&P grew about the same for twenty years, both 
under-performing the broader market. Grocery stores are hard. 
(Source: Good to Great) 


KROGER, A&P, AND THE MARKET 
Cumulative Value of $1 Invested, 
1973 - 1998 


Kroger: $198.47 
$200 


$150 


$100 


General Market. 
$19.86 


- A&P: $2.47 
0 —— aaa See ai 
1973 1985 1998 


Kroger exploded with success after “confronting the brutal 
facts,” while A&P continued to falter. 
(Source: Good to Great) 


Collins tells a fantastic story, abridged here: 
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A&P stood as the largest retailing organization in the 
world and one of the largest corporations in the United 
States, at one point ranking behind only General 
Motors in sales. Kroger, in contrast, stood as an unspec- 
tacular grocery chain, less than half the size of A&P 


A&P had a perfect model for the first half of the twenti- 
eth century ... cheap, plentiful groceries sold in utilitar- 
ian stores. But in the affluent second half of the twenti- 
eth century, Americans changed. They wanted ... big- 
ger stores, ... fresh-baked bread, flowers, cold medi- 
cines, forty-five choices of cereal, and ten types of milk. 

. and they wanted to do their banking and get their 
annual flu shots. In short, they no longer wanted gro- 
cery stories. They wanted super-stores. 


Here’s what’s interesting: Both Kroger and A&P were 
old companies heading into the 1970s [Kroger 82, A&P 
111]; both companies had nearly all their assets invest- 
ing in traditional grocery stores; both had strongholds 
outside of the major growth areas of the United States; 
and both companies had knowledge of how the world 
around them was changing. Yet one of these companies 
confronted the brutal facts of reality head-on and com- 
pletely changed its entire system in response; the other 
stuck its head in the sand. 


Collins goes on to detail this last sentence—how both 
companies were fully aware of these changes, A&P even 
opening a test store under a different name, which was a 
“super-store” that didn’t even sell A&P items, which out- 
performed their standard stores. Not wanting to face 
those facts, they shuttered the test store and went back 
to business-as-usual. Meanwhile, Kroger made the same 
experiments, found the same results, and decided to piv- 
ot the entire company to become the modern-day 


supermarket. 


A&P saw the truth, but failed to face the truth. Whether 
it’s Jim Collins analyzing dozens of companies or Rumelt 
with his experience with dozens of strategies, the pattern 


is the same: Face the truth, or be killed by it. 


When it’s hard, we avoid it 
It seems obvious that avoiding the truth would lead to 
bad outcomes, so why do we do it? Because it’s hard. It’s 


not the only obvious thing in our lives that we avoid, 


solely because it’s hard. Diet? Exercise? Giving feedback? 


Working on our relationships? 


It’s hard to tell the truth to someone’s face. It’s hard to 
realize that your industry has completely shifted, and it’s 
really hard to say that out loud in front of the whole 
company. It’s hard to say “no” to a customer when you 
have bills to pay, and it’s hard to make a major strategic 


choice, because what if you’re wrong? 


That’s a good explanation for failure to face the truth, 


but it’s not a good reason. 


Face the truth. 


Go ahead. Faith will follow.” 
—Jean-Baptiste le Rond d’Alembert, 

encouraging early practitioners of calculus, 

even though rigorous proofs of its legitimacy 


were still a hundred years away. 


Printed from: A Smart Bear 
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The “Great” Product Manager, a.k.a. the Impossible Product 


Manager 


by Jason Cohen on April 23, 2022 


According to the Internet, being a Product Manager is 
impossible. Can you ever measure up? No. But don’t worry, 
there’s a better answer. 


“What you see here is the culmination of countless 
suggestions from employees, focus groups, and 
consultants. Consequently, it’s terrible.” 


What makes a Product Manager great? The prolific 
Shreyas Doshi! gives us the list of requirements in_a 


tweet-storm: 


1] highly recommend following @Shreyas, despite the following 
critique; he is a font of wisdom I have bookmarked and swiped 
many times. 


e Great PMs consistently and singularly improve the 
company’s trajectory. 

e Great PMs are masters of the art of blending 
quantitative and qualitative inputs, as warranted by 
each individual situation. 

e Great PMs become the worldwide experts in their 
domain. When new to a domain, great PMs bootstrap 
this process by seeking the counsel of existing 
worldwide experts. 

¢ Great PMs are diligent about using a variety of user 
research methods to inform what product to build in 


the first place. 


Great PMs also listen to what isn’t said [by 
customers] and anticipate where the industry overall 
is headed when developing their product hypothesis. 
Great PMs know that buy-in isn’t enough; you need 
passion & ownership to build great products. Great 
PMs facilitate discussions that get the entire team to 
come up with creative product ideas. 

Great PMs understand task leverage and spend the 
majority of their time on the highest-leverage tasks 
for the company. 

Great PMs, in the rare instances of product failure, 
improve not just their own approach but they also 
share the lessons learned with the broader company. 
Great PMs are adaptive—they have a wide repertoire 
[of tools and processes and workflows] that they 
expertly tweak for each specific team’s needs. 

Great PMs are outstanding problem preventers. Great 
PMs are discerning about which problems to prevent, 
which problems to solve, and which problems not to 
solve. 

Great PMs edit the company’s product ethos—they 
identify the unintended flaws in the principles, fix 
the flawed parts—and only then follow & espouse it. 
Great PMs know that career ladders are imperfect 
proxies: they’re more fixated on tangible competence 
& impact than on checking off boxes on the ladder. 
Great PMs also learn through work projects, but they 
learn a lot more about their craft in their personal 
time because of their curiosity & passion for self- 
improvement. 

Great PMs ultimately decide what’s best for users & 
the business. 

Great PMs ensure the product strategy is optimal. 


Great PMs work hard but are rarely overwhelmed. 
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Haha, that last one is funny. Don’t be overwhelmed, but 
also “learn a lot more about your craft in your personal 


time.” 


These recitals always include a little “out,” an opportuni- 
ty for the writer to wriggle out of responsibility for com- 
manding the impossible. Something like, “The final rule 
is: You can break any rule, if you know what you’re 


doing.” In this case, the 31st tweet: 


Naturally, very few PMs are Great. 


And I don’t know any Great PM who does all of the 


above, all of the time. 


That’s because Great PMs know that these ideas should 
be viewed as signposts, not as commandments. 


So you could be super-human at two dozen different 


things. Now what? 


I've used the following framework for Product 
Management at WP Engine to answer this question, so 
that as a whole team we in fact constitute the mythical 
omnibus “Great PM,” but we achieve it with a set of great 


—but plausible—real human beings. 


The PM decides what to build 

The role and scope of “Product Manager” varies between 
companies, but the role centers on deciding what to 
build. 


INPUTS EXECTION 


SCRUM 

SIPMECY ——~ perme —> Ppp. owner 
ee 

CUSOMER, —” COORDINATION 8 

ResEArclr COMMUNICATION 


Looking backward, there are inputs to the decision—the 
long-term vision and strategy, and an intimate and evolv- 
ing understanding of the customer; these are part of 
Product Management, because without them we cannot 


make good decisions. 


Looking forward there’s the execution of building and de- 
livering to customers, the complexity of engineering exe- 
cution together with coordinated efforts by marketing, 
sales, customer service, and external partners to deliver 
the entire experience to customers; these consequences 


of the decision are also part of Product Management. 


You could argue that this particular break-down is arbi- 
trary, however in my experience this categorization is es- 
pecially actionable for hiring, careers, and designing the 
PM organization, and is compatible with other ways? of 


decomposing the role. 


2 Product thinkers categorize PM roles in myriad combinations. 
Lenny Rachistky in a podcast says it’s three things: (a) Shape 
the Product, (b) Ship the Product, (c) Synchronize the people. 
Later in the same podcast he says it’s five skill areas: (1) 
Execution, (2) Collaboration, (3) Leadership, (4) Vision + 
Strategy, (5) User Research. In another blog post he lists the 
ten jobs of Product Management. You'll find the four areas be- 
low align with any of these break-downs; while all paths arrive 
at the same destination, I find this one particularly useful in 
working with real-world people. 


The four roles of Product Management 


Strategist 

Analyzing, crafting, communicating, and updating the 
answer to the question: How do we apply our durable 
advantages to the opportunities in the market, to win? 
What do we look like in the success-case, years from 
now? What are the most important challenges to 
achieving that? Specifically what must we do in the 
coming years to overcome those challenges? 
Competitive analysis, market analysis, customer analy- 
sis. Determining the few, key personas to focus on. 
Distilling noisy data into clear identifications of inter- 
nal strengths, external challenges, and market oppor- 
tunities. Positioning the product to maximize value 
and perceived value, and thus price and willingness- 
the right long-term results, while also acknowledging 
that not all important things are numbers, and that 
revenue and retention are outputs of great strategy 
plus great execution, not themselves a strategy or 
even an operational goal. Having the fortitude to 
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make decisions that will affect us for years, and cause 
us to say “no” to 90% of our ideas, so we say “yes” to 


exactly the right ideas. 


Customer Whisperer 

Not just listening to customers, hearing them, under- 
standing their underlying needs, motivations, even 
their emotions. Not order-taking the features they 
know how to name, but rooting out the things they 
worry about so much they would pay to decrease the 
worry, or figuring out the result they need to get to 
earn a promotion or achieve their ultimate goal, as op- 
posed to just “features.” More than figuring it out, 
wanting to figure it out. Having the “taste” to sense 
what features would be especially useful even in the 
absence of clear quantitative data, or the nuance that 
separates a serviceable UX from a delightful one that 
causes an NPS-10 customer to actually “Promote” in 
real life. Translating these insights into tangible fea- 
ture ideas or use-cases of things to build that make the 
customer a hero. 


Scrum Product Owner 

An unfortunately ambiguous phrase for the uninitiat- 
ed, the Scrum “Product Owner” owns the work-back- 
log, and works with engineering to execute it. Owning 
the work from start to finish, breaking down epics into 
stories, prioritizing and scheduling innovations for 
customers against engineering demands against re- 
quests from other teams against the broader needs of 
the company. Working tightly with engineering not 
just to shake them upside down until estimates fall out 
of their pockets, but working together to solve the 
puzzles of “most value (whatever that means) in the 
least time (or risk),” together finding the 80% solution 
that will take 20% of the time, so that we deliver 
quickly and learn quickly. Using the tools of story 
points and velocity and retros for team-wide construc- 
tive introspection and improvement, fulfilling the re- 
sponsible aspect of being “self-managed.” Stretching 


to maximize how much we deliver, but practical and 


celebratory so that constant striving doesn’t turn into 
death-march burn-out. Being in command of the prod- 


uct and team. 


Orchestrator 

Delivering the “Whole Product,” as Geoffrey Moore 
defines it, means not delivering working code only, 
but also enabling others: Marketing to grab attention 
and spark curiosity, sales to convert potential energy 
(leads) into kinetic energy (sales), customer service to 
shepherd the customer through good times and bad, 
enabling the Marcomm teams for PR and events, and 
external integrators and consultants who operate on 
behalf of clients. Tracking and communicating status, 
the good news and the challenges, celebrating the 
wins and facing the challenges with crisp articulation, 
calming stakeholders in the knowledge that the team 
is fully aware of what they need to overcome, and 
thus in command even in dire situations. Running 
great meetings—starting with a clear goal and how 
the discussion will support the strategy, tight agendas 
so participants come prepared and use synchronous 
time wisely, with decisions made and_ recorded. 
Engaging the whole company on occasion, with inspir- 
ing presentations at all-hands meetings so everyone is 
excited about we’re accomplishing together, and so 
that the teams working directly on the product feel 


proud, and feel seen. 


ha 


“Fire good. All like fire. But sometimes fire not good. 
That why Og make exciting new product. Og call it 
‘Bucket of Water.” 
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The conclusion is not that “a Great PM must be the mas- 


ter of all this.” 


In fact, exactly the opposite. 


One excellent, one good 


In my experience, echoed by a few people I’ve chatted 
with, there is a general rule of thumb vis-a-vis the cate- 


gories of work above: 


A “Great PM” is 
excellent in one area, 
good in at least one other, 


and doesn’t have time for more than two. 


Using “Great PM” language, you should strive to be 
“Great,” as superlative as possible, in one of those four 


areas, and pretty darn good at a second. 


I find this particular categorization of four job-areas 
aligns well to strengths and abilities of real people. 
Whatever you ascribe to nature or nurture, instinct or ex- 
perience, for each of those job-areas I can immediately 
recall specific people who are excellent at nearly all com- 
ponents within a single area, but variously good or lack- 
ing in components of a second, and then completely the 
wrong temperament or lack of desire to excel in others. 
These four “buckets” seem to me to be natural divisions 


of ability and proclivity. 


I don’t believe in magical unicorns who are excellent in 
all four areas? But suppose they do exist, and you hired 
them. Would it be physically possible to execute well in 
all four areas? It’s easy to see that it’s impossible, if you 


add up the time it takes to do great work. 


3 Perhaps there are, but the probability that you'll find one, or 
hire one, or build a whole organization of them, is near-nil, al- 
though you will surely find a lot of candidates who imagine 
themselves to be one. 


Strategy is on-going—markets change, competitors 
change, customers change, new data comes to light (usu- 


ally only after expending considerable time acquiring it), 


and whenever you’re “pencils down” on one iteration of a 
strategy, there are always eight more things you still 
haven't figured out yet. Customer discovery takes a long 
time—scheduling and running interviews, compiling re- 
sults, mapping what you heard, prioritizing activities or 
opportunities or use-cases (depending on your frame- 
works), trying to (in)validate assumptions with data. 
Being a Product Owner takes at least twenty hours per 
week to write great stories and run Scrum ceremonies. 
Anyone who has program-managed multiple teams 
across multiple departments knows that scheduling and 
organizing and cajoling and status-updating and data- 
gathering and meeting-preparation takes a lot of time. 
Even with sensational skills, no one has time to do it all 


(with excellence). 


We are not going to breed a new 
race of super-humans; we will 
have to run our organizations 
with the humans we have.” 


—Peter Drucker 


While one person cannot be all things to all people, 
across the entire product team, you do need excellence in 
all four areas. Thus the manager of a Product organiza- 
tion isn’t simply a “manager,” but rather an “organization 
designer,” choreographing this outcome through inten- 


tional hiring. 


You'll often find these roles in titles other than “Product 
Manager.” Communications and collaboration is often 
done by a Program Manager—a role where if you think 
it’s not useful, it might be because you haven’t worked 
with someone who is great. Marketing, sales, and sup- 
port enablement is often done by Product Marketing (i.e. 
somewhere under the CMO rather than somewhere un- 
der the CTO). With highly technical back-end teams with 
little or no customer exposure, a sociable engineer or ar- 
chitect can be the perfect Product Owner. Mature organi- 


zations might have UX Research teams who are expert in 
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customer analysis—building data-based personas, craft- 
ing interviews, compiling results, and pooling insights 
across multiple product teams who happen to share 


customers. 


As the organization designer, your job is to “assemble ex- 
cellence” in all areas. Even incremental improvements 
here can be impactful, because so often organizations 
have excellence nowhere. 95% of employees complain 
they don’t understand the strategy, or don’t know their 
role in it; founders-in-PM-roles often run on instinct 
more than systematic customer analysis; engineering 


teams often complain of underspecified or under-ex- 


plained stories; engineers are often surprised at the gulf 
between what they know to be true about the product 
and what Sales claims on the phone, just as customer ser- 
vice folks are often surprised at the gulf between what 
they know to be true about the product, and what engi- 


neering thinks is “a job well done.” 


Rather than unicorn-hunting, take this more practical 


route in your career, or in designing your team. 


Printed from: A Smart Bear 
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Moats: Durable competitive advantage 


by Jason Cohen on May 7, 2022 


Industries commoditize over time, delivering similar 
products at similar prices resulting in low profit. Moats are 
the antedote; your strategy must create some. 


RILEY REAL ESTATE 
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“This is what separates us from the competition.” 


Economics is useless for predicting what the market will 
do tomorrow, but it is prescient at predicting long-term 
forces that shape entire industries. Business strategists ig- 


nore this at their peril. 


One of these forces is that industries commoditize: 
Companies copy the best ideas from each other, whether 
in features to attract customers or cost-savings to build 
profits, so they converge to similar products with similar 
cost structures. Undifferentiated in most dimensions, the 
flexible dimension is price, and in a real-life Prisoner’s 
Dilemma, price reduces until profits are driven to zero. 
It’s great for the consumer—the best products at the low- 


est prices—but bad for the companies. 


To avoid homogenization, a company needs differentia- 
tion that others cannot copy. Best is differentiation that 
customers care about, because that should lead not only 
to higher prices (because customers cannot get the same 
value elsewhere) but also greater market share. Second- 
best is differentiation in cost structure, because this al- 


lows the company to convert those savings either into 


profit (keep in the bank, or distribute by dividend or 
stock buy-back) or growth (spend in sales, marketing, or 


new product development). 


Durable differentiation is rare, especially in the software 


industry where almost anything can be copied. 


It is not even enough to have a competitive advantage; 
the advantage must also be durable. Snapchat invented 
the idea of “Stories,” but Facebook and Instagram copied 


it, so the advantage didn’t convert into market share: 


Facebook's Snapchat Clones Have 500M Users Each 


Worldwide DAUs of Instagram Stories, WhatsApp Status, Facebook Stories and Snapchat 


@ Snapchat ©@ Instagram Stories 


Acree 500m 
@ WhatsApp Status @ Facebook Stories 500m Mar 2019 
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Facebook doesn’t invent anything; they copy. But there is no 
“first mover advantage” unless that advantage is also durable, 
i.e. not easily copy-able by rivals. 


A taxonomy of moats 


The most important thing to me is 
figuring out how big a moat there 
is around the business. What I 
love, of course, is a big castle and 
a big moat with piranhas and 
crocodiles.” 

—wWarren Buffet 
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Durable advantages are often called “moats,” after 
Warren Buffet. The concept appears everywhere—a char- 
acteristic of fundamental truth—for example more re- 


cently and famously in Hamilton Helmer’s Seven Powers. 


Jerry Neuman collected, organized, and detailed a terrific 
taxonomy of moats. Such a reference is useful when ana- 
lyzing other companies (which moats are they building?) 
or figuring out which moats are best for your business to 
construct. 


Moats 
(Structural Deterrences to Competition) 


State Granted Special Know-how Scale System Rigidity 
| Favoritism — Policy Closely Held rIncreasing unit value + Intra-company 
L Patents U Trade secrets Network effects Business model innovation 
ae User networks Value chain innovation 
[ Resuaton Two-sided marketplaces ‘Disruption’ 
} Licensi 
Peseeee Platting: Company culture 
Tacit i i 
ats nak Decreasing unit costs L Ecosystem 
tig Experience curve Sunk costs K 
Charters ‘ oer Switching costs 
+ Legal monopolies Procedural knowledge [Economies of scale Tying/bundling 
} Targeted support Tacit knowledge Msi of scope Rae 
L state champion Institutional memory dvertising 
‘Customer insight Discounted supply Gornplementary assets 
— Control! of mership ; : eRe 
Scarce Land "Variance reduction Societal 
Resources | L Gawmaterats Cost of capital Traditional 
oor Internal hurdle rates Cultural 
‘Willingness to experiment Nationalistic 

tracts Religious 
L with Owners 
L with suppliers Moralistic 


A real-world example will breath life into these trite bul- 
let points. 


Applying moats to a commoditized 
market: Cloud Computing 


AWS (Amazon Web Services) was the first and is still the 
largest “cloud computing” provider. Its industry is an ar- 
chetypical example of a market that commoditizes. 
Indeed, all computing infrastructure markets that preced- 
ed the cloud also commoditized, from individual compo- 
nents (e.g. RAM and disks) to entire physical data cen- 
ters (with power, internet connectivity, climate-control, 


and physical security). 


Cloud computing—paying only for what you need, by the 
hour, by the megabyte, scaling up and down at will—is a 
discrete leap forward in customer value, but commoditi- 
zation is inescapable, as every by-the-hour CPU, gigabyte 
of disk, and gigabyte of network transfer from one ven- 


dor is the same as from another. If renting infrastructure 


is the only strategy, the only dimension remaining to 
compete on is price, so price falls, and the industry 


commoditizes. 


Indeed, that’s exactly what has happened, for example in 


storage: 
Cloud Storage Costs Over Time 
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Different types of storage have different prices, but each type 
has identical price across vendors. All prices are low. 


But AWS has largely avoided commoditization. The proof 
is in the market-share and in the profit. Every year since 
2014, AWS’s profit has been more than half of Amazon’s 
entire profit; in 2022 it’s generating $21B of profit at a 
30% operating profit margin, and growing: 


Amazon Web Services, 6-year financials 
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If it were possible for a competitor to steal business by 
undercutting prices, they would. Indeed, they have; com- 
petitors like Google, Microsoft, and Alibaba charge less 
for similar service. Google in particular spends billions 
more dollars per year than they make, plus they have all 
the requisite “smart people,” but even at scale in 2022, 
on annual revenues of $20B for their cloud computing 
services, they’re still losing nearly $4B annually. None of 
this is hurting AWS’s profits, demonstrating that AWS has 


not commoditized. 


Diverging Fortunes 


Operating profit/loss for Google Cloud since the end of 2018 and for AWS from mid-2015, when both 
companies were generating nearly the same revenue* 


$6 billion 
5 
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When both companies Year 2 Year 3 Year 4 
generated the same revenue 


Why not? We might expect the answer is “moats,” but 
what are they? 


AWS Moats 

In 2019, then-CEO of AWS (now-CEO of Amazon) Andy 
Jassy, on stage in a live interview with the great Kara 
Swisher, was asked directly: Why hasn’t AWS commodi- 
tized? Jassy immediately fired off the moats from their 
anti-commoditization strategy, annotated here with the 


bullet points from the taxonomy diagram: 


Innovates the fastest [Economies of Scale + 
Willingness to Experiment] 

Jassy says they use their size to fuel more and faster 
innovation. Specifically, revenue today plus line-of- 
sight to revenue growth for tomorrow, means it is easy 
to justify a massive investment in innovation. The 
platform with the most innovation, will have the best 
tools for developers, which they believe is the key to 


winning. 


Biggest Ecosystem [Complementary Assets] 

The biggest vendor will attract the most 3rd-party sup- 
port, which in turn is vital for developers who must in- 
tegrate with other tools, especially in the modern 


world where everything is a SaaS. 


Advanced operating maturity [Learning Curve] 
Jassy says, “There is no compression algorithm for ex- 
perience.” Even Microsoft and Google, throwing bil- 
lions of dollars and thousands of engineers at the chal- 
lenges, are not able to overtake AWS. 


There’s another moat that Jassy didn’t list, probably be- 
cause he didn’t want to talk about market-share) and 
possibly because it could be interpreted as gently insult- 


ing to their customers: 


l Later in the interview, he refused to answer basic questions 
about how much market share AWS has, or the relative posi- 
tions of Microsoft, Google, and Oracle, even under pressure 
from Kara Swisher, the excellent interviewer. Being the market- 
leader invites regulation and puts you on the defensive in anti- 
trust cases, so public companies shy away from such claims; for 
example Amazon owns 43% of all eCommerce at the time of 
this writing, but they publicly position themselves as a general 
“retail” company, so that their share is “only” 6%. 


Brand of “The Leader” [Brand] 

The old adage was “no one was ever fired for picking 
IBM.” Now that’s true of AWS. The idea is that if you 
pick the market leader, your career isn’t in jeopardy if 
it doesn’t go well, because you picked the least-risky 
choice. Often “the leader” is also “more expensive,” so 
this is not only good for long-term differentiation, but 
for profitability. 


There’s another moat that is clearly part of every major 
cloud provider’s strategy; perhaps Jassy didn’t mention it 


because it’s slightly nefarious, as it involves lock-in: 


Has the biggest menu of technology to select from 
[Switching Costs] 

If customers only buy commodity services like CPU, 
RAM, disk, and network, it’s easy for them to leave for 
a cheaper cloud. So, AWS has created hundreds of 
“services” covering every imaginable corner in the 


world of infrastructure: big data storage and retrieval, 
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web analytics engines, AI algorithms, logging, alert- 
ing, change-management, source code repositories, 
network firewalls, and support for dozens of open 
source software packages, all for rent by the hour or 
gigabyte. Furthermore, they encourage use of these 
services by making each one free when each is only 
lightly used. A software developer who avails herself 
of this incredible array of services is more productive 
than a large team of developers would have been ten 
years ago, however her Faustian bargain is that she 
gives up her ability to switch to a competitor, which 
means as her project grows and the pricing tiers kick 
in, she’s locked in. 


Moats are integral to strategy 


Moats don’t appear quickly or by accident; they require 
consistent investment and prioritization over a course of 
years. This is exactly what a “strategy” is supposed to or- 
ganize, and what “just be agile and react to whatever 
happens” will not. Therefore, a good strategy will have 
identified one or very few moats that the company will 
create, and the primary themes of action that will exca- 


vate them and fill them with Buffett’s piranhas. 


The example of AWS shows that the taxonomy is only a 
starting point. The details of exactly how one item ap- 
plies to a specific company are essential. A strategy that 
states “We will create switching costs” is not a strategy; it 
is a statement of what you hope will become true, rather 
than an explanation of how you will cause that to be- 
come true. A strategy to intentionally create switching 


costs looks more like this: 


Objective 
Create permanent customers through high switching- 
costs, caused by extensive integration with their technol- 
ogy _ stacks, 


development workflows, and _ IT 


administration. 


How 
Maximize the number of cloud services each customer 


uses. This, in turn, requires: 


1. Create the most number of services as quickly as 
possible, so there’s more to integrate with. Must have 
more than the competition, and a reputation of 
“innovating faster than anyone else,” to attract 
customers in the first place, and so that customers 
don’t look elsewhere if we don’t have their favorite 
service quite yet. 

2. Create compelling “on-ramps” so that services are 
actually used (e.g. free tiers, great documentation, 
pre-integration with other APIs, “bundled offerings” 


that includes many APIs) 


Consequences 


1. If we build many services, quickly, some of those 
services will turn out to be duds, or might be 
strategic only as a part of the larger offering, not 
high-growth or even profitable on their own. This is a 
“cost of doing business” and will not constitute 
evidence that a team has failed or that the strategy is 
failing. 

2. Many individual services will be commoditized, 
especially when we're just reselling existing open- 
source projects. We still need to create these, because 
integrating many of them into a single customer 
project is necessary to fulfill the objective, and 
customer discovery shows that they specifically want 
to buy those products from us. Creating genuinely 
unique services are more valuable and defensible, so 
we need to do that as well, but they also take 100x 
more effort. 

3. Given that (1) and (2) demand a large quantity of 
software, built quickly, this is a costly bet; we require 
$X billion dollars over three years, plus a scaled-up 


hiring process. 


4o0f 5 


Moats: Durable competitive advantage 


The moat is named, but also how it will be built, along 
with second-order consequences (high-cost, and frequent 
failures of usage or of differentiation on a case-by-case 
basis) that we not only accept, but expect, in service of the 
objective of creating that moat. With these details and 


decisions in place, it is a strategy, not a hope. 


Make sure your strategy is explicit about which moats are 
being constructed, and the major multi-year activities re- 


quired to construct them. 


“I told you to order the piranhas earlier.” 


Printed from: A Smart Bear 
https://longform.asmartbear.com/moats/ 
© 2007-2023 Jason Cohen WW @asmartbear 
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Productive meeting activities: Leverage the team, empower the 


individual 
by Jason Cohen on May 24, 2022 


© MAZ ANDEZSON, WWW.ANDEZTOONS.COM 


ANDERSON 


“OK, now that we all agree, let’s all go back to 
our desks and discuss why this won’t work.” 


We all hate the meeting where we prove once again that 
a committee is a beast with twenty legs and no brain, or 
when we could’ve just read the materials quietly to our- 
selves at a more convenient time (and maybe we did, 
and it’s being read to us again), but there’s also the ela- 
tion of espirit de corps and accomplishment following a 
productive meeting, where in actual fact “all of us were 


smarter than any of us.” 


Even when an enlightening discussion ensues, sometimes 
the person who requested the discussion get foiled. A 
person wants to solicit ideas, but ends up getting told 
which idea is best. A person wants help deciding between 
two vetted options, but instead receives seven new 


options. 


How can we leverage the wisdom of the crowd, while en- 
suring that decisions and other responsibilities continue 
to reside with an individual? 


Some answers follow. Enjoy! 


Ranting 

I need to get something off my chest, in a trusted envi- 
ronment, where I can say things that make me vulnerable 
(e.g. “I don’t know where to start” / “I’m scared to face 
this”), or might sound frightening to others (e.g. “I don’t 
think what that other team is doing is right” / “’m doubt- 
ing my strategy and I’m worried how my team will react 
if I broach it’). 


I don’t want you fix the problem, I just want you to un- 
derstand the problem. I wouldn’t mind if you supported 
me while I work through the problem. For example, I 
have “people problems” in my organization; I don’t need 
everyone to jump in, but I do need them to “be kind” to 
my teams while we work through this, and understand 


that we might need extra time to do our work. 


This can lift a great emotional weight off the person, while 
increasing team cohesion, trust, vulnerability, and empa- 


thy, therefore it is a fantastic use of meeting time. 


Ideation 

I need ideas. Possible causes of a problem, possible solu- 
tions of a puzzle, more options for actions I could take. 
I’m stuck right now, too close to the problem, and I don’t 
have ideas, or I’m tired of looking at my ideas and I need 
new ones. I sent an email about this ahead of time, so the 
group has had time to think about it. 


I don’t want you to solve the puzzle or decide which op- 
tion is best, I just want you to help me see options that 
I’m not currently seeing. For example, I have a strategic 
challenge where I need to solve for two apparently-con- 
tradictory things, and I need ideas for how that might 
work. I don’t want you to decide my strategy for me, but 


help me see more possibilities. 
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Everyone gets stuck; no one can have all the great ideas by 
themselves. Having a space where ideas can flow but no one 
is trying to tell you want to do, is fun for everyone and im- 
possible to do alone, therefore it is a fantastic use of meet- 


ing time. 


Decision-Support 

I need help deciding something. I know what my options 
are, and I’ve analyzed them (sent ahead to the group), 
but it’s complex and I’m uncertain which to pick. I might 
even be uncertain how to decide. It could be that the best 
option isn’t yet in the list, so it could be good if we syn- 
thesize something new. But ultimately I don’t want seven 


new ideas; I want to make a decision. 


I don’t want you to tell me what to pick, but I do want in- 
put, whether on the evaluation of the options, or on what 
basis I should make a final decision. I might even ask the 
room to vote; not to dictate my future, but because I 
trust the room’s opinion and I want it as another input in 
my decision process. 


No matter how many prioritization articles you read and 
ROI rubrics you make and multi-dimensional maximization 
charts you make, complex decisions are difficult. Seeing 
your choices reflected in others’ experiences and points of 
view can help, and cannot be done alone, therefore it is a 


fantastic use of meeting time. 


Sparring Practice 

I need a Devil’s Advocate to sharpen my work. I want 
find the weaknesses in my strategy, my feature concept, 
my keynote presentation, my reorg proposal, my new UX 
design; I need a safe space where people “attack” my 
work, to surface legitimate flaws, or to reveal ways of 
making my argument that deflect potential attacks before 


they begin. 


I don’t want you do my work for me. If you have an idea 
for improvement I’d love to hear it, but I will decide what 
to do with that idea. It’s OK if you “attack” from a per- 
spective that isn’t even accurate, because my future read- 


ers might have the same misunderstanding, and I want to 
be clear even to them. I want to be challenged with the 


intention of making my work bullet-proof. 


Whenever you see a brief, tight, bullet-proof document, it 
took a lot of work to get there. No one by themselves can 
think of all the ways that others will (mis)interpret ideas, 
or think of all the legitimately good counter-arguments that 
deserve answering, therefore it is a fantastic use of meeting 


time. 


Declare the conversation 

It’s counterproductive when someone wants a Ranting, 
but receives a Sparring Practice. Declare what kind of 
conversation you want, then everyone can participate in 
the right way, and facilitators can reroute the conversa- 
tion when it inevitably strays from the parameters of the 


discussion. 


There is an “I” in Team 


Notice that everything here is selfish—each exercise is 


about what one person needs. 


This is contrary to how we often work together. We say 
things like “We need to decide on our Q2 priorities,” in- 
stead of “Person X will decide the Q2 priorities by Friday, 
so let’s discuss to ensure everyone is heard, and the best 
ideas are incorporated.” 


It’s healthy for X to seek a consensus, and X often gen- 
uinely wants to be influenced by team mates who X 
trusts and respects, particularly when opinions are unani- 
mous. However, a decision made by “us” is owned by “no 


” 


one,” when in actual fact it is owned by X. 
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Acknowledging this reality is useful, even as you also— 
correctly!-spend 98% of the conversation discussing as 


equals. 


This is also a healthy pressure against top-down control, 
in cases where the decision is supposed to reside in a 
self-managed and empowered team. That is, suppose per- 
son P is responsible for some project, and asks for help in 
a room containing peers and person Q, who is P’s boss. 
The right way for the entire room to help P are tech- 
niques like the above, where P leverages the brainpower 
of the room, but stays in command, keeps the responsi- 
bility, keeps the autonomy, even with Q present. It is a 
wonderful experience for a leader to assist without 


usurping. 


Let’s put more ideas for constructive meeting activities in a 


Twitter thread! 


© MAZ ANDEZSON, WWW.ANDEZTOONS.COM 


AnDERSoN 


“It seems I've run out of sports analogies, 
so meeting adjourned.” 


Printed from: A Smart Bear 
https://longform.asmartbear.com/productive-meeting-activities/ 
© 2007-2023 Jason Cohen WW @asmartbear 
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The "Talk vs Walk" workshop 


by Jason Cohen on June 8, 2022 


We invented this strategic exercise at WP Engine - engaging 
both Marketing and Product, generating actions for both 
sides that make products more desirable and competitive. 


fe ANTE RSON 


“Go back and give that part a little more oomph. 
Really make me believe!” 


Despite the insipid rhyming, it’s a surprisingly useful 


question to analyze: 
You talk the talk, but do you walk the walk? 


Whether it’s the early-stage company, understandably ex- 
aggerating its product’s qualities on the home page in a 


” 


lived experience of “Fake it ’till you make it’,” or the ven- 
erable company whose marketing department isn’t taking 
enough credit for legitimately world-class qualities, be- 
cause they've become so second-nature that it doesn’t oc- 


cur to them to brag about it. 
A Again with the rhyming? 


Plotting items on a “talk” vs “walk” chart is not only use- 
ful in analyzing current-state, but also in uncovering 
ways that both the Marketing and Product groups can 


sell better and create better products. 


The method 


Write-storming ideas 

Using a whiteboard? set up a vertical axis called “Talk: 
How much/loudly do we claim this” and a horizontal 
axis called “Walk: How well do we pay this off in 


practice.” 


< Likely virtual nowadays, e.g. Miro, LucidChart, or even Google 
Slides; something where folks can easily create and drag 
around “cards,” and see the results immediately. 


MAIN 
POSITIONING 
s SOMETIMES 
Ee SAY 
NNER 
MENTION 


CANNOT MINIMAL 
po 


WALK 


ACCEPTABLE. WworRLd- 
CLASS 


Then, independently write-storm?: Create “sticky notes” 
and place them wherever each person believes they 
should go on the chart. You could even allow several 
days, giving people time to think and to schedule the 
thinking during periods when their brains and work-envi- 


ronments are most effective at tackling such projects. 


3 Do not brain-storm items one-by-one. In live brain-storming 
the conversations follows the fastest and loudest people, rather 
than leveraging everyone’s brains to invent different ideas. Live 
brainstorming filters out ideas that take longer than a few min- 
utes to think of. Live brainstorming eliminates the possibility of 
having an “ah-ha” moment in the shower. “There is not a single 
published study in which a face-to-face brainstorming group 
outperforms a _ [write-storming] group” observes Leigh 
Thompson in Creative Conspiracy. Recent research suggests that 
people are less creative in generating ideas over video-confer- 
encing versus in-person; write-storming restores the creativity 
because it’s independent, even if the discussion is over Zoom. 
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Where do ideas come from? Because the exercise is about 
“what our company claims,” obvious sources are the pub- 
lic website, advertisements, public presentations and we- 
binars, quotes from your employees in the media, sales 
materials, and support materials like knowledge-base ar- 
ticles or copy/paste shortcuts. If you’re quoting directly, 
put the phrase in double-quotes to emphasize that “we 


'” 


really do say this 


There are also the things you’re doing that you’re not 
talking about. Those things are more difficult to gener- 
ate. Sources are what you're maintaining (i.e. existing 
systems owned by some team), work from recent sprints 
(i.e. work you’re actually doing), long-term planning doc- 
uments (i.e. larger initiatives), and prompts like “What 
are we really good at?” and “What technology assets or 
human capabilities have we built up over time?” 


The hardest things to think of, are the things you’re not 
talking about and also not doing. Isn’t that an infinite 
number of things? Yes, but it’s useful to list things you 
reasonably could say or do, but are intentionally avoid- 
ing. One source is backlogs of features requests from cus- 
tomers, support, sales, marketing, product, and engi- 
neers; whether you plan to do them eventually or not, 
they are “items of interest.” Another source is key, differ- 
entiating things that competitors are saying or doing 
(whether you’re avoiding those things purposefully or re- 
luctantly). Yet another is your own long-term plans for 


big initiatives you haven’t gotten to yet. 


Debating placement is one-third the value 

Come together as a group, take each idea in turn, and de- 
bate where it should go on the chart. Coalesce duplicates 
into a single card; if the duplicates were originally strewn 
all over the chart, that’s even more indication that a dis- 
cussion is useful. If there’s immediate widespread agree- 
ment on the right location for a card, skip the discussion 


to save time. 


Generally you'll find three sources of disagreements. All 


three are useful, but in different ways: 


Lack of knowledge 

“We don’t talk about this.” “Yes we do, it’s in all the 
sales materials!” “In support, we tell people the oppo- 
site.” “We say this in sales but not on the website.” 
“We used to say that, but not since the latest website 


revamp.” 


This is a great excuse to get everyone up to speed 
about what we’re saying, and where. If the answer 
varies (e.g. “we say this a lot on the home page, but 
nowhere else”), you can place the item more in the 
middle of the chart, but add a “comment” to record 
that detail; it will be useful later when you decide how 


to act on this information. 
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Disagreement on terminology 

People often interpret the same phrase differently. If 
you can’t agree on what it means internally, imagine 
how confused customers are! Or a new employee who 


isn’t part of the echo chamber? 


An example will illustrate how this is useful: In a re- 
cent exercise at WP Engine, multiple people put “En- 
terprise-Grade Security” on the board. We talk about 
this constantly, so no one disagreed that it was high on 
the “talk” axis. But there was disagreement on “walk,” 


ranging from the middle to the far-right. 


But how is this controversial? We’re well-known for 
having the best security in the industry. Our customers 
include some of the largest security companies and fi- 
nancial institutions in the world, both of which main- 
tain the highest bar for security; we are certified with 
multiple accreditations; we leverage myriad security 
products covering every layer of the tech stack; we 
commission third-party pen-tests for every new prod- 
uct and periodically for all products; we conduct an- 
nual company-wide training to guard against social 
engineering attacks; we have formal written security 
policies that we actually enforce. With all this, how 


could you not agree “Security” is 100% on “walking 
the walk?” 


Turned out, the person who placed it in the middle 
was specifically thinking about a few items in the 
backlog that would make a certain corner the platform 
even more secure. We’d never experienced an incident 
because of, and there was no reason to believe we 
would. Still, this person thought, If there’s still a few 
obvious items in the backlog that we could do, then 


we're not yet fully “walking the walk” on security. 


A discussion resolved the difference of opinion. We de- 
cided that “great security” isn’t defined by never hav- 
ing a ticket in the backlog; after all, nearly all software 
updates contains security patches (check the version 
history!). That doesn’t mean the software wasn’t al- 


ready secure. Indeed, it’s the opposite: Being so dili- 


gent that we even “fix” things that aren’t broken, 
thinking a few steps ahead, proactively preventing in- 
cidents, is what makes us secure; it’s a sign that we are 


in fact “walking the walk.” 


The group finally agreed that the ticket belongs on the 
far right side. The realization, however, was helpful 
both for the engineers who have a better mental mod- 
el of “what security means,” and for the marketers 
who have more stories to tell about how excellent our 


security is. 


Disagreement on target persona 
Sometimes you realize that a subset of your customers 
would agree that you’re “walking the walk,” but others 


would disagree. 


For example, perhaps the performance of your product 
is excellent for most customers, so when you say “fast 
user interface that never gets in your way,” most peo- 
ple would agree. But maybe 2% of your customers 
have huge amount of data that slows your UI to a 
crawl. Furthermore, that 2% are important to you— 
they might be your reference customers, or the ones 


paying you the most money. 


What do you do in these fractured circumstances? A 
simple technique is to place the item in a sort of 
“weighted” location. So e.g. if 80% of your customers 
(by revenue) would consider this completely “walking 
the walk,” but 20% would say it’s pretty low, perhaps 
you place the item at 80%. However, add a comment 
to the sticky-note explaining this detail; it will be use- 
ful later when you decide what actions you want to 


take. 


Another technique is to split the item. So e.g. “Fast UI 
(for customers <100 properties)” and also “Fast UI 
(for customers with =100 properties).” Then place 


them independently. 


If you have different, distinct target personas, you 
might find a lot of the cards are being split. In this 


case, consider making two different charts altogether, 
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i.e. one for “Talk vs Walk for Persona A” and the other 
“Talk vs Walk for Persona B.” This is easier to read and 


to act on. 


Taking action 


Just having these discussions is valuable, but most of the 
value is in doing something with the resulting chart. 


In general the question is: Is it useful to take action to 
shift cards in various directions? It’s useful to consider 
this by quadrant, because those items have different sta- 


tus quo interpretations. 


CANNOT MIN(MAL ACCEPTABLE woelp- 
OASS 


WALK 


Quadrant A: Exaggerations 

These are the original source of the line “you talk the talk 
but you don’t walk the walk.” When you're “faking it ’till 
you make it,” but still just faking it. What to do? 


Do nothing (ethically) 

We understand our claims are tenuous. We should en- 
sure that our exaggeration is not unethical, e.g. claim- 
ing a medical benefit that does not exist, or charging 


for a feature that does not work. 


Live up to your own expectations (shift —) 

This message is resonating, and is a critical to our po- 
sitioning. But if it’s so critical, we have to back it up 
with reality! We'll build and ship specific things in the 
next six months that will allow us to shift these sticky- 


notes rightward to meet our customers’ expectations. 


Don’t write checks you can’t cash (shift |) 

We're attracting the wrong sort of customers—people 
who expect something we don’t deliver. As a result 
they waste time in customer service and then cancel 
anyway—a net loss for us, and a terrible experience 
for them. We’re not even getting useful product feed- 
back, because we’re interviewing people who aren’t 
our target persona. Let’s stop saying these things. 


Quadrant B: The trusted brand 

Trust is earned by repeatedly keeping your promises; 
that’s what’s happening in this quadrant. This is your 
brand, because it’s what you say and also what customers 
actually experience. This is a strength, because every- 


thing is constructively aligned. What to do? 


Do nothing 

We're doing well here, but doing incrementally better 
will not win us more customers (i.e. Kano “must-be” 
or “indifferent”). Therefore, we should track our per- 
formance on this item as a KPI, but rather than contin- 
ue investing to maximize it, we should only promise 
that it won’t fall below some minimum threshold of 
acceptability. This ensures we won’t unintentionally 
lose this advantage. Instead, focus our attention on ac- 


cumulating more strengths, or mitigating some issue. 


Leverage strength for sales (shift t) 

Can we make even more fuss about how awesome we 
are? More confident competitive positioning, lever- 
aged in advertising, more content marketing on the 
subject, emphasized more in sales processes, high- 
lighted more in the product UI. We might even be able 
to make sales by emphasizing one strength above all 


others. 


Double-down on strength (shift —) 

It’s often wise to invest more in what’s already work- 
ing, especially early in a product’s life when it’s hard 
to have even one thing that is special, or a reason to 
buy despite its many deficiencies. This only works 
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when customers value the additional investment (i.e. 
Kano “attractive” or “performance”), but if that’s the 


case, it might be the most strategic thing you can do. 


Pivot (shift |) 

While it’s unusual to jettison a strength, it makes 
sense if the company is pivoting: Changing its target, 
or how its brand and product is positioned against the 


competition. 


Quadrant C: Speak no evil 

Often there’s not much in this quadrant, because it’s easi- 
er to think of things you’re saying and doing, and these 
are neither. However, it’s valuable to list “things we know 
are weaknesses” or “things we’re saying ’no’ to,” especial- 
ly if it’s a common topic of conversation at the office, or 
if a competitor is successfully talking and walking that 
item. 


Do nothing 

This is the correct answer for nearly everything here. 
It’s expensive and high-risk to turn a weakness into a 
strength; your energy is probably better spend execut- 
ing on your current promises and doubling-down on 
strengths. 


Turn it around (shift —) 

If this is truly a key area that is critical to succeed in 
the market—which is another way of saying 
“strategic”’—then this is a weakness to highlight, and 
to invest in. Should you shift it up (start talking pub- 
licly) before you’ve shifted it right (pay off the 
claims)? Probably not: You tip your hand to competi- 
tors, allowing them to react before you can defend. 
You also set incorrect expectations for customers; later 
when you really can pay off your words, you’ve al- 
ready trained them to disbelieve you. Conversely, talk- 
ing first can freeze the market* but only if your mar- 
ket leadership position is so strong that a press release 


actually does change market behavior. Unlikely. 


4 “The U.S. District Court ... suggested barring Microsoft from 
making vaporware announcements because doing so can al- 
legedly freeze the market and discourage buyers from purchas- 
ing competing products. ... ‘I feel very sure there have been 
many times when Microsoft has announced products to freeze 
the market, [but] lots of companies have done that,’ Kertzman 
said."—Computer World, 1995 


Quadrant D: The strong, silent type 


If you are excellent, why not get credit for it? 


Do nothing 

There might be a good reason for not making a fuss. It 
could be a trade secret that you don’t want competi- 
tors knowing about (but you could brag that a trade 
secret is part of the magic that makes you different). It 
could be “boring” internal operational stuff (but cus- 
tomers might be comforted knowing that you’re oper- 
ationally advanced, and a tech blog is a recruiting 
tool). 


Talk about it (shift t) 
Even if this is a secondary or tertiary message, it’s nice 
to arm sales with more material, or test it in advertis- 


ing, or make a landing page for the subset of cus- 


tomers who would be impressed. 


Application to competitive strategy 


Turn the tables on your competition to reveal more 


insights. 
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Adding the competition to both charts 
Build the Talk/Walk chart for a competitor. 


Place a copy of your cards onto their board. There might 


be a lot in quadrant C (“Speak no Evil”). 


Then create the new cards that makes sense for them. 


These prompts help generate ideas: 


e¢ What do they emphasize (big or bold font) on their 
home page, “pricing” page, and “about us” page? 

e What main features do they highlight on their 
“features” page? 

¢ On what points do you lose to them in sales? Which 
are they legitimately great at, versus which are they 
just really good at selling? 

¢ On what points do you beat them in sales? For which 
do they deserve it, versus which are you just good at 
claiming? 

e¢ What do their customers say about them (e.g. on 
Twitter or testimonials)? 

e« What do your customers say about them (e.g. when 
complaining or asking for features, do they reference 
the competition?) 

e If you were making an investor pitch on their behalf, 
what would you say? How much of that would be 
true, versus aspirational? 

e¢ What cultural or moral principles do they or their 


founder or their CEO publicly espouse? 


Whatever new cards you add to their chart, also add to 
your own chart, even if it mostly creates a pile in quad- 


rant C. This is where it gets interesting. 


Analyzing the competitor’s strengths 
It’s especially instructive to examine your competitor’s 
strengths. On your board, highlight the competitor’s 


Quadrant B cards. Everything here is actionable: 


Quadrant A: Exaggerating against their strength 

Youre both making a claim, but only your competitor 
is fulfilling that promise. Over time this erodes your 
brand; potential customers start dismissing you, say- 


ing “Yeah, everyone claims to have X, but only P really 


does X.” You were already claiming something ahead 
of having it, and now the competitor is exposing your 
little fibs. 


Before you decide to knuckle down and match the 
competitor strength-for-strength, critically consider 
whether you want to run a race where you're starting 
out behind. Do you have what it takes in long-term 
will-power, and investment in time and skill, not only 
to catch up to where they are today, but to where they 
will be in another year? Will the result of that expen- 
sive and risky activity pay off enough to be worth it? 
Occasionally the answer is “yes,” but usually the an- 
swer should be “no, let’s move this card downward— 
stop making this claim—and instead focus on increas- 
ing our differentiating advantages in Quadrant B, 


where we are the ones in the lead.” 


Quadrant B: Table-stakes; find the differentiation 
You’re both doing well here. Perhaps these topics are 
considered “table-stakes” for any competitor in the 
market. That’s not a problem, but neither is it an ad- 
vantage for you. Table-stakes are important to main- 
tain, but it’s a mistake to over-invest in them at the ex- 
pense of things that differentiate you, or at the ex- 
pense of the few key weaknesses that you’re convert- 
ing into strengths. 


Your comparative advantages are the cards in this 
quadrant that aren’t shared with the competitor. Do 
you have a significant set of those cards? If not, in- 
venting some might be the single most strategic thing 
you could do. 


Quadrant C: It’s not your thing, and that’s OK 
This is where your competitor is differentiated against 
you. It’s easy to get discouraged or think this is a prob- 


lem; it’s not. The only problem is if you don’t have 
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your own unique strengths. If you both have your own 
strengths, that’s just defines your respective niches, 


which is useful to recognize. 


As with Quadrant A, it’s highly unlikely that you 
should attempt to “turn it around” and become strong 
where your competitor is already strong. But, if there’s 
one item that’s truly important to win, you could set 
that intention. Just know that you can’t have more 
than one or two of those at a time, and often you'll 
only get to mediocre, not a strength. It surely must be 


treated as a Rock. 


Quadrant D: Don’t give them a free pass 

Youre letting your competitor take all the credit, even 
though you’re just as good. Change that! Sure, you'll 
probably just match them, not create a new differenti- 
ated advantage, but since you’ve already done the 
work to be excellent in this area, don’t give them an 


uncontested win in sales calls. 


I hope this exercise is as useful to you as it has been for 


us! 


Printed from: A Smart Bear 
https://longform.asmartbear.com/talk-vs-walk/ 
© 2007-2024 Jason Cohen yy @asmartbear 
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Fermi ROI: Fixing the ROI rubric 


by Jason Cohen on June 20, 2022 


Traditional rubrics fail to reveal the best answers, or how to 
explain those answers to others. After explaining why, the 
following system solves both failures. 


“You've been traded for some big data, 
two spreadsheets, and an algorithm.” 


“Delivering maximum value in minimum time” sounds 
wise, but it’s not exactly a stunning insight. It’s not like 
we're trying to take an eternity to deliver trivialities. 


It is more analytical—but no more insightful—to insist 
on “Maximizing ROI.” Return-On-Investment is some 
measure of “value,” divided by some measure of “time,” 


yielding some measure of “productivity.” 


With good intentions and this reasonable-sounding goal, 
we reach for the rubric: A spreadsheet of inputs of nu- 
meric “value” and “time,” a calculation of the metric of 
productivity, and an output where the best ideas are sort- 


ed to the top. 


Invisible failures of the rubric 


Unfortunately, the rubric fails us for several reasons. The 
failures are not obvious, so we don’t realize they’re hap- 
pening. Explaining the traps will lead to a framework 


that avoids the traps. 


“ROI” contains more noise than signal 
Consider a feature that ended up producing 20% less im- 
pact and taking 50% more time than originally expected 


—a common outcome in the real world: 


Impact Effort ROI 
Estimated 60 4 15 
Actual 48 6 8 


In this reasonable scenario, it ended up producing half 
the ROI than originally calculated. Therefore, in 
retrospect, the initial ROI estimate was largely noise. 


Apply this observation to a typical ROI rubric: 


Impact Effort ROI 


Feature A 34 2 17 
Feature B 60 4 15 
Feature C 36 3 12 
Feature D 10 1 10 


The spreadsheet suggests (A) is the winner, and in 
particular that (A) is clearly better than (D). But neither 
of those statements are true. If (A) could easily result in 
half its stated ROI, there’s no objective justification to 


claim its superiority over (D), or over any other item. 


Shocked at this revelation, we could react by ignoring the 
lesson. We could justify (A)’s supremacy by telling our- 
selves “estimation errors cancel themselves out.” But the 
errors do not cancel out. Effort is almost always under- 
estimated; 4-sprint projects that stretch into 6 sprints are 
not “cancelled out” by an equal number of 4-sprint 


projects that are completely finished in 2 sprints. 


“Impact” is ill-defined 
No matter how you measure impact, your numbers end 


up far less precise than they appear in the spreadsheet. 
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It’s difficult to predict numbers like “revenue-increase 
due to feature X.” Teams use various techniques to ad- 
dress the uncertainty, but in every case the number in the 


spreadsheet contains significant error: 


e Revenue we would get from existing customers 
currently asking for this feature. 
That’s precise but often uncorrelated with how much 
money we will make from it in the long run, so it’s 


not an accurate measure of impact. 


e Additional sales we would make in the next year if we 
had this feature. 
This number is low-confidence, high-error. Even in 
hindsight it’s hard to attribute revenue from a single 
customer to a single feature; if it’s unmeasurable post 
hoc, surely it’s unpredictable ex ante. 


e Use arelative scale, e.g. “1-5,” calibrated by previously- 
created features. 
Defined as “1 is like feature X, 5 is like feature Y.” 
Not only is this a wild guess, different people will 
interpret the scale differently, yet answers differing 
by a single point generates a large variance in 
computed ROI. 


It’s even worse when want to combine multiple metrics 
of impact, e.g. revenue, unit-profitability, and retention. 
Each contains intrinsic error, then a mathematical combi- 
nation compounds the error, then the ROI calculation ex- 


pands the error yet again. 


You can’t tell from glancing at the spreadsheet, because 
input numbers appear typical and output numbers boast 
many digits after the decimal place. But those digits 
aren’t indicative of precision. Theyre a random number 
generator. 


The Fermi Solution 


These traps dictate requirements for a solution. We need 
scores containing dramatically more signal than noise. 


The scores must be well-defined, which means their defi- 


nition and _ estimation should be obvious and 


uncontroversial. 


Only Fermi estimations allowed 
The first full-scale nuclear bomb was detonated at 
5:29am, July 16, 1945, in the New Mexican desert of the 
United States. The physicists who invented it were hud- 
dled in a truck behind a plate of welder’s glass to reduce 
the radiation to non-lethal levels. 


The physicists were already causing trouble. Future 
Nobel Prize-winner Richard Feynman inexplicably decid- 
ed to observe the blast without eye protection, causing 
frightening but ultimately temporary blindness. Current 
Nobel Prize-winner Enrico Fermi had taken bets with mil- 
itary guards about how much of the atmosphere would 
ignite, and whether it would incinerate the entire state or 
the entire world; some of the guards asked to be excused 
from the base, angering the project director. 


“Enrico Fermi” by Argonne National Laboratory is licensed un- 
der CC BY-NC-SA 2.0. 
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Fermi was also interested in the amount of energy re- 
leased by the blast—one of the main goals of the test. 
Not wanting to wait for official analysis, he made his 
own estimate on the spot, using a technique that now 


bears his name, and that we will use to fix our rubric: 


About 40 seconds after the explosion the air blast 
reached me. I tried to estimate its strength by dropping 
from about six feet small pieces of paper before, dur- 
ing, and after the passage of the blast wave. Since, at 
the time, there was no wind I could observe very dis- 
tinctly and actually measure the displacement of the 
pieces of paper that were in the process of falling while 
the blast was passing. The shift was about 2 1/2 me- 
ters, which, at the time, I estimated to correspond to 
the blast that would be produced by 10,000 tons of 
TNT. 

—Enrico Fermi, Top Secret interview July 16, 1945, de- 
classified in 1965 


The official estimate of the energy output of the blast 
was 21,000 tons of TNT. Fermi’s estimate was surprising- 
ly accurate given such inaccurate input data and quick, 


simple, mental calculations. How did he do it? 


The trick—useful everywhere in life—is to estimate val- 
ues using only orders-of-magnitude, a.k.a. powers-of-ten. 
No “low/high ranges,” no precision, not even any digits 
other than a 1 followed by a quantity of Os. It sounds far 
too imprecise to be practical, and yet Fermi’s bits of pa- 


per demonstrate that it just might work. 


Joel Spolsky famously loved an interview question built 
for Fermi estimation: How many gas stations are there in 


Los Angeles County? “I don’t know,” although accurate, 


fails the interview. Fermi estimation, succeeds. 


e There are 10 million people in the LA area (or at 
least, more than 1 million and less than 100 million, 
so by the rule of “only powers-of-ten,” our number is 10 
million. The actual number in LA County happens to be 
10 million). 

e 1car per person (or at least, not 10 and not 0.1. The 
actual number is 0.54). 

¢ A person refills 1 time per month (the actual number 
is closer to 2.5/month, but it’s not 10 and not 0.1). 

e A gas station handles 10,000 refills per month (200 
per day yields 6000 per month; it’s not 1000 nor 
100,000). 

e With 10 million refills per month (10 million cars 
with 1 refill per month), divided by 10,000 refills per 
station, 1000 gas stations are required. 


Despite being wildly inaccurate in detail, the end result 
of 1000 gas stations is indeed the nearest order-of-mag- 
nitude to the actual number of 600. Surprisingly accu- 
rate, considering we arrived at an answer in a minute or 


two, without looking up a single number. 


What would happen if we used only Fermi estimates in 


our ROI calculator? 


An immediate result is that most numbers are trivial to 
estimate. Note how easy it was to decide the values in 
the gas station example, because the two adjacent power- 
of-ten choices were definitely wrong. It’s nice to be so 


confident! Relish that. 


Another result is that we’ve satisfied one requirement 
generated from the failures of the rubric: Scores are ei- 
ther identical or very different, causing calculation error 
to be small in comparison. The values are largely obvious 
and uncontroversial, if only because our artificial con- 
straint on their values makes the adjacent choices absurd. 


This is progress. 


Sometimes the numbers are still controversial, e.g. Could 
this product produce $1M in the next few years, or actually 


$10M because it would dramatically increase both reten- 
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tion and our ability to win new deals? But this debate is a 
wonderful use of our time, because it means people have 
wildly different assumptions, or different levels of opti- 
mism, or a different understanding of the customer, the 
market, or the idea. So another constructive result of 
Fermi estimation is that we spend our debate-time on 
strategic-level discussions, and no time on areas where 
we disagree in small degrees that are in any case over- 


whelmed by typical estimation-error. 


In exchange for this progress, we lose precision. And ad- 
mittedly, sometimes we do possess precise input data. For 
example, maybe we could look up the fact that the aver- 
age number of cars per person is 0.54, and use that fig- 
ure instead of 1. Surely it’s wise to retain precision 
whenever possible? Yes, but only if you recognize that, 
after doing math with other Fermi estimates, the end re- 
sult is still not more accurate than the power-of-ten. That 
is, even using 0.54 instead of 1, as soon as you com- 
bine it with your extremely inaccurate estimate of 10,000 
refills-per-station, you still have to zero-out all the digits 
in the result except the first one. You must meticulously 
discard the noise; if you’re diligent about that, your esti- 
mation is more accurate. Otherwise, maybe you should 


just enjoy the simplicity of the basic Fermi system. 


So we've satisfied the first requirement. What about the 
final remaining requirement, which is to clarify the defin- 


ition of “impact?” 


Fermi Impact: Quantitive edition 

Fermi estimation solves the puzzle of how to estimate the 
revenue impact of a sizable feature. We can pick any defi- 
nition we want, but limit ourselves to Fermi estimation, 
and suddenly it becomes easy, or at least a simpler dis- 


cussion. Any of these definitions could work: 


e This feature will increase revenue by: $1k/mo, 
$10k/mo, $100k/mo 

e This feature will increase new customers/month by: 
1, 10, 100, 1000, 10,000 

e The take-rate of this feature in our existing customer 
base would be: 1%, 10%, 50% 


e The number of customers who would actively use 
this feature would be: 0.1%, 1%, 10%, 50% 


We cheated a little in those last two, because “100%” is 
not a possibility. Still, a 5x separation is pretty good; the 
danger is when the difference is only 2x or 20%. This is 


still a much wider spread then, say, Fibonacci estimation. 


It’s clear how this works with quantitative measures, but 


what about important things that are not numbers? 


Fermi Impact: Qualitative edition 
How you score things which aren’t numbers like “brand 


development,” “competitive advantage,” or “customer 
delight?” 


The following is a real example from a few years ago, 
when WP Engine was launching a new product line 
called Atlas! We decided to build a thought-leadership 
presentation that would engage software developers on 
topics relevant to the new product line. We wanted it to 
be genuinely intriguing and useful. We brainstormed 
twenty-two topic ideas, all pretty good at first glance. 
How should we select just four or five topics, given that 


“ntriguing” and “useful” are qualitative and subjective? 


1 Tagline: The complete Headless WordPress Platform for ab- 
surdly fast dynamic sites. 


MIZE ZEALIZES HE’S ONLY 
BZOUGHT HIS ‘Cc’ GAME 
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First, we decided on our goal, even if impossible to mea- 
sure, or not even a number. The primary goal was for 
everyone to come away loving the content and inter- 
ested in our software. We broke this down into four 
sub-goals, where the overall goal would be achieved only 


to the extent that all four of the sub-goals were achieved: 


1. Topic is widely applicable 

2. Topic matters to real people 

3. Topic is insightful (i.e. non-obvious to a typical 
practitioner) 

4. Topic is relevant to specific features or capabilities in 


Atlas (i.e. self-promotion) 


The next step was to convert these concepts into some- 
thing we could “score,” i.e. specific questions that a per- 
son could contemplate and debate with another person. 
Otherwise, two people will have different ideas of what 
each goal means. The answers still won’t be numbers, but 


at least the questions are specific: 


1. How many customers care about this topic? 

2. How much do those customers care? 

3. How insightful is our perspective on this topic? 

4. How powerfully or uniquely does Atlas pay off the 
insight? 


Because the answers aren’t things you can measure with 
numbers and units, we had to use some sort of “1-5” 
scoring system or, to take the Fermi lesson, powers-of-ten 
numbers. The key is to be specific as to what each number 
means, otherwise two people will have different interpre- 


tations for a phrase like “Insight is a 2 out of 5.” 


1. How many customers care about this topic? 


Fermi Value Definition 
100 Definitionally everyone 
70 Most 
30 Some 
0 Ican think of one or two 


0 Noone 


2. How much do those customers care? 


Value Definition 
1000 Mission-critical to the success of their project 
100 “I care; this is a serious concern we’ve discussed 
internally” 
10 “I’m curious, tell me more” 
1 “Meh, whatever” 


0 “Don’t waste my time with this” 


3. How insightful is our perspective on this topic? 


Fermi Value Definition 
1000 “Wow!!! This changes everything.” 
100 “I took notes, thanks, that’s a great point.” 
10 “Yup, OK, makes sense.” 
1 “No shit, Sherlock.” 


0 “No clue what you're even talking about.” 


4. How powerfully or uniquely does Atlas pay off the 
insight? 


Fermi Value Definition 
1000 “Whoa, Ill buy Atlas just for that alone!” 
100 “Ohhh, nice, ok Pll take a look.” 
10 “That’s fine.” 
1 “So what?” 


0 “I actively do not want this.” 


The combination of Fermi values and specificity in both 
the questions (rather than broad concepts) and values 
(rather than “1-5”) made it easy to score. Because “all 
four” scores were required to deliver on the original goal, 


multiplying the scores” sorts the best to the top. 


2 The mathematically-inclined reader might point out that be- 
cause all values are powers-of-ten, the operation “multiply 
everything and sort” yields exactly the same result as “Number 
these 1-5, then add everything and sort,” because the latter is 
just the logarithm of the former. Why, then, bother with Fermi 
values? (a) As in the first question, specific values (as opposed 
to strictly only powers-of-ten) might have a valid meaning, 
which creates different values from “1-5.” This “breaks ties” in 
a meaningful way. (b) By emphasizing that the values are de- 
signed to be widely divergent, we generate better specific 
phrases, and more agreement in scoring each topic, and there- 
fore hopefully a more accurate output. 
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It worked; the presentation was routinely scored “5S out 
of 5” in post-event surveys*, and the biggest complaint 
was a desire for even more detail. That is a “complaint” 
indicative of success: It means “I want to hear more from 
you,” and therefore is further evidence that the choice of 


topics was spot-on. 


3 Oh the irony of scoring the presentation 1-5 after everything 
we just said! 


Fermi time estimates 
We've explored how to estimate “impact” with Fermi-ap- 
proved values, but what about time-estimates? Those 


also exhibit large errors. They, too, need the Fermi 
treatment. 


“So, where do you see yourself in ten minutes?” 


Nearly twenty years ago, before Scrum defeated rivals 
like Extreme Programming to become the de facto stan- 
dard for waterfall-averse software developers, our engi- 
neering team at Smart Bear Software used a simplistic 
but effective method of estimating work. Our software 
was installed on-premises by customers (this was The 
Time Before SaaS), so “continuous deployment” was im- 
possible. Instead, we made a few major releases per year, 
supplemented with minor bug-fix releases. We planned 
about four months of work at a time. 


We scrawled each idea on an index card, with only 
enough space for a title and a few bullets about scope 
and intent. We placed time estimates in the bottom-right 


corner, but the only three choices were: 2d, 2w, 2m— 
two days, two weeks, or two months* Whatever you 


think the real estimate is, round up. 


41 hear you yelling “No! Story Points!” Hang in there, it’s OK, 
this was long ago in the Third Age when the Elves still abided 
in Middle-Earth. Elves don’t use story-points. 


The overwhelming majority of cards generated no dis- 
agreement over the estimate; this saved us hours of 
analysis and debate. The controversial ones were always 
a matter of definition and scope, which is a typical and 
important conversation to have, regardless of your sys- 
tem for estimating work. 


To select the items and plan team capacity, we spread the 
cards across a table, and each person held the set of 
cards they were going to execute. The project timeframe 
was measured in weeks; a typical window was 16 weeks. 
We used a rough conversion that two 2d cards equals 
one week, and one 2m card equals eight weeks”. It was 
easy to see who was over- or under-scheduled, and then 


load-balance according to capacity and ability. 


> You might wonder why we didn’t just use a mathematically 
identical scale like 0.5w, 2w, 8w or some abstract unit of “ef- 
fort” that could scale like 1, 4, 16, or a Fibonacci number of 
weeks. The reason for using real time is that we felt it was easi- 
er to imagine than abstract units. The reason for keeping the 
quantity of values to three is the Fermi reason: This minimizes 
controversy and forces you to think critically about scope. 


If a project required multiple cards (nowadays we'd call 
that an “epic” or “big rock”), we marked that set of cards 
with a color. The idea is that either all the cards of that 
color should be accepted, or none, otherwise we would 
have spent a lot of time on something without having 


shipped a usable feature. 


Here’s the punchline: It typically took a few hours to cre- 
ate the entire plan, and four months later we typically hit 


the plan within a week of the original estimate. 


In retrospect, it’s a form of Fermi Estimation. It’s not ex- 
actly powers-of-ten, but measured in work-days it’s close: 


2 vs 10 vs 45. Essentially powers-of-five, it still main- 
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tained the key idea of Fermi—to have so few choices that 
the correct one is easy to identify—while also being 


chunks of time that a human can relate to. 


Putting it together 


Let’s combine these ideas, do the math, and decide 
whether it’s helping us make better decisions. 


Single-valued impact 

Suppose we’re picking features to implement, and the 
only metric of value is the revenue we believe that fea- 
ture will generate in the next twelve months. We'll con- 
vert “effort” into work-days as given just above, and 
round ROI to the nearest 10% to make it easy to read. 


Here’s a typical result: 


NTM Revenue Effort ROI 


Feature A $100,000 2m 2200 
Feature B $10,000 2w 1000 
Feature C $1,000 2d 500 
Feature D $10,000 2m 200 
Feature E $1,000 2w 100 


This has some nice properties: 


Biased towards more impact 

Products A, B, and C are each examples of “takes more 
time, but generates more impact.” The rubric prefers 
more impact in every case. This is a good bias, espe- 
cially since impact in reality is often less than we had 


hoped. 


Still correctly ordered if we mis-judged 

Consider what happens if we misjudge impact by 20% 
and effort by 50%—the original hypothetical from the 
beginning of this article. If Feature A delivered only 
$80,000 and took three months, the actual ROI would 
be about 1200, still ahead of Feature B. Or if Feature B 
delivered $8,000 in three weeks, actual ROI is 533, 
still ahead of Feature C. Therefore, typical misjudg- 
ments are not changing our decision. We fixed that 


deficiency. 


Easy to explain to others 

It’s important to be able to explain your decisions 
crisply. The typical ROI rubric does not; the explana- 
tion is that one item slightly edged out another item, 
which isn’t confidence-inspiring, and doesn’t sound 
like a strategic decision. In contrast, Fermi ROI is easy 
to summarize; for example, why Feature A was select- 
ed: Feature A has by far the greatest potential for im- 
pact. The impact is so large, it’s worth spending multiple 
months on it. Or why Feature B wasn’t selected: 
Feature B would be faster to implement, and therefore is 
a great idea, but Feature A’s impact is so much larger, it’s 
still the better choice. If we get part-way into Feature A 
and realize it’s much harder than we thought, or much 
lower impact than we thought, then we’ll pivot into 


Feature B. 


Multi-valued impact 

In the “presentation” example, we had four impact 
scores, which we multiplied together. The reason it made 
sense to combine them in that way, is that all four are re- 
quired to achieve the goal (as explained previously). 
When you have a case where multiple factors are needed 
together to achieve a singular goal, then multiplying is 


the correct way to compute the score for that goal. 


A common mistake is to use this formula when there are 
scores for things which are not related. For example, sup- 
pose you’re wanting to maximize revenue, and also cus- 
tomer delight. You could make a Fermi score for each, 
but how do you combine them to get an ROI? 


There are two answers, depending on your philosophy: 


Do not combine unlike attributes 

There is no way to compare “revenue” with “delight.” 
These are different units of measurement, so any com- 
bination is nonsensical. Instead, decide which of the 
factors is most important, and compute the ROI of just 
that. Then, if there is a tie, you can break the tie by 
comparing “delight.” This is also easy to explain after 


the fact: Features A, B, C are are equally effective at 
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generating revenue, which is our most important goal. 
Among those, Feature B increases customer delight the 


most as well, so we’re implementing B. 


Add instead of multiply (only when attributes are 
equally important) 

Although it is arguably a lack of strategic decision- 
making to claim that multiple attributes are all equally 
important, it can make sense if you are scheduling sec- 
ondary features. That is, suppose you’ve already 
scheduled the most important strategic items, and 
now you're “filling in” with “quick wins” where you 
truly don’t care in what manner each is a “win.” Here 
you add the values instead of multiply. This way, a 
huge ROI in one area wins (but we don’t care which 
area). Getting some value in two areas, is not as good 
as getting a lot of value in one area. This final point is 
satisfied when you use Fermi values, but traditional 


rubrics get it wrong. 


Breaking ties 


The Fermi-style rubric has another interesting property: 
There are only a few unique values of ROI. In the “rev- 
enue” example above, there are only 3 realistic numbers 
for revenue, and only 3 values of effort, so only 9 unique 
combinations of ROI. 


The good news is, the computation clearly separates the 
best from the rest. The bad news is, you can easily make 
ties. In the “presentation” example earlier, in real life we 
had two topics in clear first and second place, but a 
three-way tie for third. We didn’t have time for all five 
topics, so we had to break the tie. 


These ties are not a failing of the system. Just the oppo- 
site: It means we are able to identify ties, rather than al- 
lowing noise to trick us into believing one is the 


“winner.” 


“IT remind you that I recommended against upping our game 
while simultaneously taking it to the next level.” 


Still, we have to break the tie. There are several ways to 


do it, while preserving the advantages of this method. 


Adding precision or intentional bias 

It’s tempting to add “precision” to your input values, and 
indeed that might break the tie. But beware of falling 
back into the accuracy problem and the explanatory defi- 


ciency of typical rubrics. 
You should add precision only if: 


1. You are certain the precision is accurate, or 
2. You are creating an intentional bias, e.g. to over- 


weight one factor relative to another. 


There’s a mathematical trap in option (2). It’s tempting 
to “weight” a factor by multiplying it by a constant, e.g. 
“we'll double this score, so it counts twice as much as 
other scores.” However, this actually doesn’t do anything 
at all, because it just means all totals for all items are 
multiplied by 2; this doesn’t change the ordering, and 
doesn’t break ties. 


Instead, what matters are the sizes of the intervals be- 
tween the choices. That is, if scoring for this item was 
originally 1, 10, 100, you could reduce how sensitive 
the total is to this value by diminishing the differences 
between values, e.g. by going up by powers of three in- 


stead of powers of ten: 1, 3, 9. 
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You also don’t need to keep the spacing regular. You 
might rank more options as a 0 to effectively “filter out” 
things that don’t meet a desired standard. You might de- 
cide that the two options at the top of the scale, that 
originally were 100 and 1000, should be more like 800 
and 1000, reflecting a bias that “both options have es- 


sentially the same value, one just slightly more.” 


The run-off 


Given a set of items that we’ve agreed are “equal in ROI,” 


pick a new dimension to break the tie. 


Time-estimates are no longer part of the process. We al- 
ready agreed these are efficient in terms of impact-ver- 
sus-time, so now we focus only on some additional di- 
mension of value. This simplifies the process, which is 
not only nice for decision-making, but also for explaining 
the decision. 


If you don’t want to evaluate the items along a new di- 
mension, you could eliminate some of the existing di- 
mensions. For example, in the “presentation” case, we 
decided that, all else being equal, we would rather select 
topics where we have a really interesting insight, rather 
than topics that are great at selling the product. 
Therefore, we performed a run-off using “insight x 


reach,” ignoring the other values. 


To determine which is the most important dimension to 


maximize, using this guiding question: 


If we maximize this dimension in the next few months, 
then even if we get nothing else accomplished, it will 
still be a success. Conversely, if we moved other nee- 
dles, but this dimension remained unchanged, it will 
have been a failure. 


The human factor 
A different way to break the tie is to include something 
else that you value, but that shouldn’t be used to make 


the primary decision. 


Since the primary decision is a tie, we’re “allowed” to in- 
dulge ourselves in something else, even something un- 


connected to “business metrics” in the usual sense. 


Because we’ve already maximized a business metric, you 


could break ties using something completely different: 


Team Excitement 

Whether because it’s fun, interesting, or the team is just 
tired of hearing certain customer feedback and really 
wants to do something about it, “excitement” is an excel- 
lent, under-appreciated attribute. Excited teams are hap- 
pier, feel more fulfilled, feel more listened-to, and work 


harder and better. It is the elusive “win-win.” 


Since “excitement” is an ambiguous term, one way to 


convert to a Fermi value is the following: 


Fermi 


Excitement Definition 


0 No one is excited to work on it. 
1 One person is excited to work on it. 
10 About half the team is excited to work on it. 


100 All, or all but one, is excited to work on it. 


Confidence 
Confidence is hard to measure, but you know it when 


you feel it (or don’t feel it). It might not be correlated 
with risk—just because you are confident, doesn’t mean 
you are correct. Still, faced with two choices, where 
you’re sure you know how to execute the first, but with 
the second youre full of doubt, it’s logical to pick the 
first. 


Since “confidence” is an ambiguous term, some people 
use some sort of risk percentage, e.g. “I’m 60% sure we 
can do this.” But pinning down a number is suspicious, 
and debates don’t seem fruitful. One way to convert to a 
Fermi value is the following: 
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Fermi 
Confidence Definition Good luck! 


0 There’s no way we can pull this off. 


1 Imean we can try, but don’t be surprised when it 


fails. Printed from: A Smart Bear 


https://longform.asmartbear.com/roi-rubric/ 
goes very wrong. © 2007-2023 Jason Cohen ‘W @asmartbear 
100 This is completely within our domain of exper- 


10 Wecan probably do this; we'll be surprised if it 


tise, we’ve successfully executed something simi- 
lar in the past, and nothing in the task-break- 
down is uncertain. 
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Binstack: Making a maximal multi-dimensional decision 


by Jason Cohen on July 2, 2022 
Binstack is a technique for selecting the “single most 


impactful” solution when there are multiple, incomparable 
dimensions to evaluate. 


SS: 

=3 
ee 
ts 


ANDERSON 


“Do it, don’t do it. We don’t 
care anymore.” 


Many decisions in life and business are instances of “mul- 
ti-dimensional maximization,” in which we wish to pick 
the “single best” among a set of choices, but we’re con- 
founded because each choice is variously better or worse 


along different dimensions. 
Examples: 


e Which major feature should we spend the next six 
months building? 
(P would generate revenue, but Q would reduce 
cancellations, but R would save us money) 

e Which candidate should we hire? 
(P has the best skills, but Q has more experience in our 
market, but R seems like the best culture-fit) 

e Which new marketing campaign should we spend 
thousands of dollars to test? 
(P is cheaper to try, but Q has a larger reach, but R is 
targeted at our industry) 


Not only do you need the best decision, you also need to 
be able to explain your decision to others, especially to 
those who wish the decision had gone a different way. Do 


not under-value the importance of crisp explanation. 


The “rubric” is the typical framework for these decisions; 
a separate paper on this site explains how to use one ef- 
fectively for “ROI-style” decisions! Unfortunately, while 
it may feel productive to fill many cells with many num- 
bers, and while it may feel analytically rigorous to convo- 
lute those numbers into a final score, this fails to clearly 
identify the best choice, and fails to create a clear expla- 


nation for the choice. 


1 The goal of ROI is to maximize efficiency, i.e. deliver the most 
amount of value per unit of time. This paper asks a different 
question: How to decide the single most valuable thing, regard- 
less of cost, with incommensurate and conflicting dimensions 


After demonstrating and analyzing the causes of this fail- 
ure, I present an alternative framework I’ve nick-named 
“Binstack2” 


2 Additional resource: Adam Waselnuk created a Notion Process 
Template for Binstack. 


Why rubrics don’t add up 


Consider two players in a game, with attributes: 


Attribute Player P Player Q 


Health 8 5 
Strength 5 7 
Speed 2 6 
Endurance 5 2 


Which player is better? If one scored higher than the 
other in every dimension, the choice would be simple. In 
this case, each player is better than the other along two 
dimensions, and worse along two; objectively there’s no 


clear winner. 
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So let’s try a rubric. In its simplest form, we add up the 
scores contributed by each dimension, and the total score 
decides the winner. Unfortunately, this doesn’t tell us 


which one is better: 


Attribute Player P Player Q 


Health 8 5 
Strength 5 7 
Speed 2 6 
Endurance 5 2 
Total: 20 20 


Games often engineer this result on purpose, to create 
players that are different but not over-powered. This 
creates a balanced game, but to make a confident 
decision about which one is “best,” we need something 


imbalanced. 


Real-life rubrics often result in a pile of options that 
share a similar score, resulting in no clear winner. Even if 
we make a decision, we can’t explain the decision to oth- 
ers, because in actual fact it’s a tie, and the tie was bro- 


ken arbitrarily. That’s no way to make a decision. 


To create separation, people often add “weights” to the 
raw value to create a new kind of “score.” In this case, 
suppose that with our game-playing style we don’t care 
much about Speed, but we can really leverage Strength. 
So we assign weights which we multiply against the 


scores, to compute a customized metric of “value.” 


Attribute OrigP OrigQ Wt WtedP WtedQ 


Health 8 5 1.0 8 5 
Strength 5 7 2.0 10 14 
Speed 2 6 0.5 1 3 
Endurance 5 2 1.0 

Total: 20 20 24 24 


Well, shoot. That didn’t help. What if we contrive to force 
Q to be better, by intentionally using weights that 


penalize the two attributes where P is superior? 


Attribute OrigP OrigQ Wt WtedP WtedQ 


Health 8 5 0.5 4 25 
Strength 5 7 2.0 10 14 
Speed 2 6 0.5 1 3 
Endurance 5 2 0.5 2.5 1 
Total: 20 20 17.5 20.5 


Even with a conspiracy to throw the election for Q, it 
wins by a mere 2.5 points out of 20—hardly a 
resounding victory that would give everyone confidence 


in the decision. 


This happens in the real world. Even with weights, a 
clear winner often does not emerge, and again we’re 


back to a weak, indefensible decision. 


Worse: In the real world we rarely have precise scores. 
Attributes like “potential new revenue” and “increased 
customer satisfaction” are not predictable with accuracy. 
For more qualitative measures we use scales “from 1 to 
5” which are even less precise. This imprecision creates 
inherent error, which is then compounded by multiplying 
weights. Differences in the final results might be more er- 
ror than signal. How would you even know? 


Worse again: The attributes aren’t comparable. Whatever 
units the game uses for “Health,” they're not related to 
whatever units the game uses for “Speed.” By adding 
them together, we’re implicitly saying “these are compa- 
rable,” but they are not. Weights are supposed to solve 
this by converting everything into some sort of “value,” 
but if you say out loud what the weights are doing, it 
sounds incorrect. For example: “Every $150k of addition- 
al ARR is exactly as valuable to us as 10% more customer 
satisfaction. Indeed, we would be fine with customer sat- 
isfaction going down 10% if we added $300k in new 
ARR.” Really? 


It’s largely noise, which is why we’re unhappy with the 


so-called “winner.” 
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Solution: Binstack: Stack-ranked binary 
attributes 


To transcend the noise, stop pretending that the values in 


the rows are precise or comparable. 


A general rule of complex decisions, is that often they’re 
difficult because we’ve avoided making other, smaller de- 
cisions. That is the case with the rubric, as we’re about to 


fix. 


Our top-level purpose is to pick the item that maximizes 
impact, and to be able to explain why. So, first we’re go- 
ing to make smaller decisions about the true impact of 
each item, and then we’re going to make a decision 


about which impacts are most valuable. 


Binary materiality 

No more values, no more weights, no more scores. Either 
an item materially contributes to that attribute, or it 
doesn’t. “Materially” means the effect is so large you can 


measure it easily: 


e Not just “more revenue,” but at least a 10% bump so 
that the curve visibly changes. 

¢ Not just “more retention,” but a 20% decrease in 
cancellations tied to a specific cause. 

¢ Not just “more intuitive,” but a 40% decrease in 
support tickets on a certain topic. 

¢ Not just “more competitive,” but sales will add it to 
their standard presentation and marketing will add it 
to the feature-table on the pricing page. 

¢ Not just “more profitable,” but overall gross profit 
margin will improve by 1%. 

e Not just “will pay for it,” but putting it in a higher 
pricing tier or add-on will cause 5% of customers to 
upgrade. 

¢ Not just “better UX,” but a 50% increase the success- 
rate for people moving through the interface. 

e Not just “widely used,” but 40% of customers 
surveyed scored at least 4 out of 5 on whether they'd 


use the feature. 


e Not just “customer satisfaction,” but moving from a 
4/10 to a 8/10 on a survey related to this area of the 
product. 

e Not just “thought leadership,” but marketing commits 
to getting ten external articles to reference it in the 


next quarter. 


Force people to write down exactly what the material 
change is expected to be. Not because the estimates are 
accurate, but because it forces the person to think 
through the answer. Most ideas, we’ll eventually admit, 
are so incremental that we won’t be able to measure the 
effect; that means it is “not material.” That’s a tough fact 
to face, but remember the point of the exercise is to force 
exactly these conclusions, to drastically reduce the field 
of ideas so that only the actually-best ideas remain. 
These smaller decisions will make the larger decision 


easy. 


Because this “material change” is just a guess, we won’t 
put it in an equation—no computing with noise! But if 
you can’t justify a magnitude greater than “business as 
usual,” the idea is simply not impactful enough. Your 
standards are higher than that. 


In our example, if we simplistically considered any score 
that is “6 or greater” to be “material,” we’d already have 


a winner: 


Attribute Player P Player Q 


Health v 

Strength v 
Speed v 
Endurance 

Total: 1 2 


Each “point” in this method is meaningful, so even a 
difference of 1 point crowns a clear winner. With real- 
world attributes, and a sufficiently high bar, you will 
reject nearly all items quickly. People won’t like that— 
their favorite thing will be cut—but it’s the only way to 
stop wasting your time dithering between a pile of things 


that won’t make a difference. 
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It’s also extremely easy to explain your decision: Q mate- 
rially impacts two important things; everything else is 


less impactful. 


i vii 
“T haven't yet decided if I’m staunchly in favor, 


or staunchly opposed. But rest assured, 
however I vote, it will be staunchly.” 


It is, of course, still common to have ties. Indeed, if in 
our example we considered anything “5 or greater” to be 


material, it’s back to a tie?: 


3 Although in this case perhaps the problem is that our standards 
for “materiality” are too low, as opposed to the options being 
too good. 


Attribute Player P Player Q 


Health v v 
Strength v v 
Speed v 
Endurance v 

Total: 3 3 


To address this, we need one more rule. 


Stack-ranked attributes 

Back to the top: (1) We’re trying to isolate the one thing 
that would be most impactful, and (2) complex decisions 
feel impossible because of a lack of smaller decisions. 
We’ve made some decisions already, but we need a few 


more to isolate the one winner. 


We have to decide which attributes are most important. 
Currently we are treating all attributes as equally impor- 
tant—a check mark next to “Endurance” is equal to a 


check mark next to “Health,” but is that really true? 


With a standard rubric, the fact that all attributes are not 
equally important drove us to reach for “weights.” But 
that computation confounded us with noise. Instead, we 
simply order attributes by importance, in a single ranked 


list. No numbers. 


“Simply order them” is easy to say but not easy to accom- 


plish, because people get stuck in circular debates: 


“Growth is more important than profit, because it’s pos- 
sible to optimize our costs later.” 


“Yeah, but if we’re unprofitable on a unit basis we'll 
cause a cash-crunch, so we have to be profitable first.” 


“Yeah, but if it’s only about profit, the best thing to do 
is just 10x prices, and whichever customers stay are su- 
per profitable, but that would be wrong.” 


“Yeah, but if it’s only about revenue, the best thing to 
do is to sell $1 bills for $0.80, and that would be 
wrong.” 


This conflict highlights the “smaller decisions” that still 


need to be made. 


Both sides are correct in saying it’s bad to maximize one 
thing with no regard to any consequences. But surely 
you’ve already ensured your list contains nothing out- 
right absurd* So these reductio ad absurdum arguments 
are moot and can be ignored. Assume (and enforce that) 
the ideas are sensible, then decide what outcome is most 


important. 


4 If you have trouble ensuring that items are meeting basic stan- 
dards, create criteria for an idea making it onto the list to begin 
with. Examples: it can’t be less unit-profitable than some pre- 
determined target, it can’t take longer than N sprints to exe- 
cute, and it can’t require significant retraining of the support 
team. 


A typical ordering for a VC-backed B2B company, opti- 
mizing for “growth is paramount, because if growth is 


there, we'll be able to raise more money,” could be: 
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1. Revenue growth (i.e. “the single biggest driver of 
equity-value”) 

2. Number-of-customers growth (i.e. “market share”) 

3. Product experience (i.e. “customers love of the 
product”) 

4. Support cost (i.e. “a cost; more importantly, a 
measure of usability”) 

5. Infrastructure cost (i.e. “a cost; should organically 
improve with scale”) 

6. Net-profit expansion (i.e. “profitable business 


model”) 


A bootstrapped company designed to create wealth for its 
founders and employees, while being a place where em- 
ployees genuinely love coming to work and customers 


genuinely love the product, might be: 


1. Cash-basis profit expansion (i.e. “wealth creation” + 
“mandatory to keep the business alive”) 

2. Product experience (i.e. “the reason we get up in the 
morning is building a great product people love”) 

3. Fun (i.e. “I built this business to be the place I want 
to work for”) 

4. Number-of-customers growth (i.e. “stagnation is the 
prelude to death”) 

5. Minimizing number of employees (i.e. “we joined a 
small company to avoid bureaucracy”) 


These are generic attributes; “more revenue” could in- 
clude almost anything. It’s better if your lists are more 
specific, based on current circumstances, or focused on a 
subset of the strategy. For example, suppose a product 
that targets mid-sized restaurant chains is having trouble 
with customer retention. A better list might be: 


1. Dramatic increase in usability 
The #1 reason customers give us when they cancel in 
their first year, is that training their employees is too 
difficult, so the software never gets used. 

2. Reduce costs 
Price is the #1 reason that long-tenured customers give 
when they switch to a competitor; reducing costs means 


we can reduce prices while generating the same profit. 


3. Increase market-differentiation 
If there were features that customers couldn’t get 


anywhere else, they would stay despite (1) and (2). 


Note how obvious corporate goals like “grow revenue” 
and “happy customers” are embedded in these goals, but 
insights or data produce more specific immediate goals. 
This will cause even better ideas to be selected, and will 


help the team brainstorm better ideas in the first place. 


Binstack: The final process 

You’ve finally made all the “small decisions” that make 
the big decision clear. With your stack-ranked attributes 
and binary scores of which items materially affect which 


attributes, here’s what you do: 


1. Cross out items that don’t materially address the top- 
ranked attribute. 
2. For the second attribute... 
a. If no tasks address it, move on to the next 
attribute. 
b. If exactly one remaining task addresses it, that’s 
the winner; you’re finished. 
c. If multiple remaining tasks affect it, cross out all 
the others and continue on to the next attribute. 
3. Repeat step (2) for the third attribute, fourth, etc. 


These steps honor our smaller decisions about which re- 
sults are most important to manifest (ordered attributes), 
and what these options really accomplish for us (binary 
materiality). It also ensures that we'll materially affect 
our #1 attribute; even if another idea moves several oth- 
er needles, we still have to honor our “small decision” 


about what is of paramount importance. 


The final decision is trivial to explain. It goes something 
like this: 
We decided the most important things we have to ac- 
complish in the next few months are to grow top-line 


revenue and create defensible technology. Item P does 
both of these things; none of our other choices did. 


Or defending why you didn’t pick some other item: 


5 of 7 


Binstack: Making a maximal multi-dimensional decision 


Item Q is a solid idea; indeed it would both increase 
profitability and increase our differentiation in the mar- 
ket. However, the most important thing right now is to 
grow revenue, and item P accomplishes that whereas 
item Q doesn’t. However, in future, if we change our 
priorities, or complete item P item Q will be wonderful 


to consider! 


“I like you, Anderson! You're decisive!” 


Real-life errata 


“Effort” conspicuously absent 

Many rubrics are set up as “ROI” calculators, i.e. measur- 
ing impact relative to the cost of achieving that impact. 
This often results in selecting less-impactful items, be- 
cause they are cost-effective to implement. Sometimes 
that’s the right choice, but Binstack is about selecting 


only for maximum impact, not cost-efficiency. 


If you actually want to maximize ROI, use this method. 
And here’s how you decide which method is right for 


your current decision. 


Fun is underrated. Add it in. 
It’s still possible to tie. If you have lots of items left over, 


perhaps your materiality threshold isn’t high enough; 
raise it to thin the herd. 


Supposing you have two ideas that are truly indistin- 
guishable, you could flip a coin. I don’t recommend that, 
because you can’t explain your choice. A person passion- 


ate about the choice you rejected would be upset to hear 


you ruled against them so flippantly. Instead, pick which- 
ever item the folks doing the work want to do. Do what’s 
fun. 


It still sounds flippant. What business does “fun” have in 
business? When people work on something fun, they 
work harder and better while enjoying themselves more 
—more productivity yet more happiness. Do not dismiss 
this life-hack. 


Of course we cannot do what’s fun at the expense of 
what needs to be done, but when those two things are 
not in conflict, why would you not round off in favor of 
fun? 


If you like this idea, take it further: Put “fun” in the at- 
tribute stack-rank, and rank it high. Even second position 
is not ridiculous. Knock that #1 priority out of the park 
while having fun. What’s wrong with that? 


Evolving stack-rankings, and different stack-rankings 
per team 

You should expect the stack-ranking to change over time, 
even rapidly. In the early days of a company, just getting 
any customers is hard, at any price, so that might be 
much more important than revenue or profit. A mature 
company who reliably gets customers in the door might 
be more interested in expanding efficiency or profitabili- 
ty, not because growth is unimportant, but because it’s so 
systematic that it is no longer an existential threat, and 


other things are more pressing. 


If you have multiple teams, and therefore the time to ex- 
ecute multiple items, you might want separate lists for 
different goals. For example, you might say, “We want 
one initiative that will materially increase profit, one that 
materially increases our internal effectiveness or efficien- 
cy, and all the rest should maximize growth.” Each of 
those would be the number-one item in its own stack- 
ranked list, and the attributes below that might be copied 


from the company-wide general list. 


In all cases, remember to hone the attributes with more 


specificity, to generate better ideas. 
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What if nothing is left? 


I’ve generally found that the best 
product ideas live at the 
intersection of “duh” and “holy 
shit.” 


—Aaron Levie 


Sometimes we’re so harsh with our materiality threshold 
that none of our ideas meet our exacting standards. What 


does that mean? 


It means your ideas aren’t good enough. It means your 
problem wasn’t one of prioritization after all, but rather 


of not having ideas worth prioritizing. 


Focus the team on this new, more pressing problem: To 
generate better ideas. Here’s some help with generating 


better ideas. 


But this will take time! Fine, put it in the sprint. But we 
need a plan right now! Too bad; it’s better to take a 
month to find a wonderful thing to spend the rest of the 
year on, than to plod along doing things that aren’t valu- 
able enough. Do high-ROI small projects in the 


meantime. 


Why bother scoring everything when most will be 


rejected immediately? 


Indeed, you needn’t bother. That saves time. 


However, remember that an important aspect of decision- 
making is explaining your decision. Often, explaining 
“why not Q” is just as important as explaining “why P” 
Other people will want to know that you seriously con- 


sidered other options. 


Most importantly, there are people who really wish you 
had selected Q. Maybe Q was even their idea. The rejec- 
tion will be easier to accept if Q was seriously and gen- 
uinely considered. Perhaps, by being invited into the de- 


cision process, the person who invented Q will come to 


the right conclusion on their own. This is important; do 
not underestimate the human—and humane—part of the 


process. 


Reductive 

Binstack can feel reductive—over-simplified, ignoring the 
reality of a complex world, therefore resulting in an in- 
correct conclusion. It is true that for complex problems 
like foreign policy, national economics, and climate 


change, a reductive approach is invalid. 


But for finding the right feature to build, or the right 
marketing campaign to launch, or the right bug-tracking 
software to adopt, or the right database to use for a new 
project, or the best candidate to hire, being (intelligently) 
reductive is how you transcend the noise, arrive at a 


clear decision, and explain it to others. 


This isn’t foreign policy, it’s a feature list. Make an impact 


with Binstack, and be happy! 


Printed from: A Smart Bear 
https://longform.asmartbear.com/maximized-decision/ 
© 2007-2023 Jason Cohen W @asmartbear 
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Rocks, Pebbles, Sand: How to implement in practice 


by Jason Cohen on July 17, 2022 


This complete work-prioritization framework builds on the 
simplistic “Rocks, Pebbles, Sand” analogy, adding the 
details you need in the real world. 


You know the geology-in-a-jar lesson from Stephen 


Covey: Schedule big things first, otherwise you run out of 
time: 
--2 


ff ‘< 


If you do little things first, there’s no time for big things. 


| | 2 


If do big things first, then you can fill in the smaller things. 


A common mistake is to think this applies only to the size 
of the work. That is, “Rocks” means “stuff that takes a 
few quarters,” “Pebbles” means “a few sprints,” and 


“Sand” means “less than a sprint.” 


This misses the most important point of work-ordering: 
It’s about maximizing impact by not allowing the easy or 
urgent things to crowd out the strategic things that take 
years to unfold but are more important than everything 
else combined. A thousand “quick wins” do not create 


durable advantages or fulfill a long-term vision. 


Another mistake is to think that the previous paragraph 
is the end of the story. “Schedule revenue-growth stuff, 
then maintenance updates, got it.” No. Each type of work 
requires different prioritization frameworks, has different 
goals, and hide different traps that make you unwittingly 


ineffective. 


If you pretend these differences don’t exist, your team 
will be working hard and delivering lots of code-commits 
—the appearance of “productivity’—but they'll feel like 
they’re not making progress fast enough, competition will 
start catching up, and they'll (correctly) complain that 
they can’t see how their work is connected to the 
strategy. 


The good news is: It does not take additional time to do 
it right. This is an instance of “smarter, not harder.” You 


just need the right frameworks. 


Three mindsets 


A tabular summary is trite, but it’s a handy reference: 


Rocks Pebbles Sand 
Effort =3 Months 1-4 Sprints <1 Sprint 
Maximize Impact ROI Throughput 
Outlook Long-term Short-term Immediate 
Scope Strategic Tactical any 
How Deliberate Analytical Intuitive 
Decide 
Role of Decider Observer none 
Exec 
Role of PM Driver Decider Decider 
(but engage 
devs) 
Beware insufficient over-estimat- over-thinking / 
impact ing over-planning 


ROI 


Now we'll justify and explain how to use this to 
maximum effect, each stone in turn. 
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Rocks maximize Impact 


ANDERSON 


“No, I’m working from home today.” 


Duration: 3-12 months 

Rocks take the most time; let’s call it 3-6 months. Long 
projects are not only expensive, they're also most likely 
to over-run, because theyre the most complex, contain 
the most unknowns, and have the most dependencies. So 


it’s really 3-12 months. 


Hofstadter’s Law 
It always takes longer than you expect, even when you 
take Hofstadter’s Law into account. 


Well-oiled agile teams will point out they can hit earlier 
deadlines by adjusting scope and pushing less-urgent 
items past the deadline. That’s wonderful for learning 
and customer-delight. But the leftover work still needs to 
be done, even if rearranged, so this doesn’t change the 
magnitude of the effort required to achieve the full effect 
of the idea. And anyway, there are good reasons why 


we're consistently incapable of estimating big projects. 


A team might only do a single Rock in a year—certainly 
no more than three—especially because other work also 
needs to get done. If you can do only one big thing this 


year, that thing had better be extraordinary. 


Maximize impact 

A Rock must deliver dramatic, measurable impact, not 
merely “incremental improvement.” It must be strategic, 
meaning that it must attack the most important chal- 
lenges you face, materially advancing the company down 
its unique path for winning its corner of the market, 
leveraging existing advantages to reduce risk and to 
forge a path that others cannot easily follow, and build 
new durable advantages. This is where teams most often 
fall short: Not delivering enough impact to justify their 


investment of time. 


Sadly, big projects not only over-run on time, but also of- 
ten under-deliver on impact! These sorts of predictions 
are famously inaccurate. So it’s even more imperative 
that we demand an enormous impact: That way, if we 
under-achieve, it was still worth the time. 


1 Fortunately, on occasion they can over-deliver by an order of 
magnitude or two; this is always the post hoc story of a success- 
ful company, the founders shaking their heads saying they nev- 
er believed it would be this successful. 


Crucially, and perhaps controversially: Do not maximize 
ROI” Your primary job is to execute your strategy to the 
fullest, spending the most-possible time on the most-im- 
pactful thing. If an idea is less impactful, yet also quicker 
to achieve, that is not the right choice. When things go 
worse than planned, that “less impact” turns into “incre- 
mental impact,” and you cannot spend half a year on 


something so trivial. 


2 ROI is “Return on Investment,” computed as a measure of im- 
pact divided by a measure of effort, resulting in a measure of 
efficiency, i.e. “value per sprint.” (Whatever “value” means.) 


Beware: “Maximizing impact” is harder than you 
think 

The most common problem is executing Rocks that aren’t 
impactful enough. The Rock claims to “make a 
difference,” but not enough difference, and after a few 


“. 


years, it feels like “we’re not moving fast enough” or 
“why isn’t revenue higher” even though the work from 
engineering is high-quality and stories are duly delivered 


every sprint. 
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Hofstadter’s Law applies not only to the time-estimate, 
but to the impact-estimate, and thus to the height of the 
bar that need to set: It must be higher than you think, 
even when you take Hofstadter’s Law into account. Over- 


shooting is the antidote to Hofstadter’s Law. 


Even if your ideas aren’t good enough, you will be tempt- 
ed to select the most impactful idea on the list and just 
do it. This is a mistake. It’s such a common mistake, it is 
a cliché: “Good” is the enemy of “Great.” You have to face 
the truth: Your biggest problem is a lack of a truly great 
idea, and you must solve that rather than embarking on a 
long, misguided journey. The team can do Pebbles and 
Sand in the meantime, thus staying productive, while 


also helping ideate and validate better ideas. 


Deliberative decision-making process 

Because you’re committing so much of the team’s life? 
and because you have to be so confident that the Rock is 
strategic and impactful, you need to spend time on this 
decision up-front. This is not an “agile” decision, it’s a 
strategic one. Once the direction is set, the big picture is 
clear, the mountain you want to climb is identified, then 
it is ideal to be “agile” in how you climb it. Execution de- 
tails are never certain; backtracking is necessary. But if 
you're climbing up the wrong mountain in the first place, 
being “agile” doesn’t help; the result is a team self-man- 
aging themselves into a mediocre, unfulfilling result. 
Although most decisions should be fast, sometimes they 
should be slow; Rocks should be slow, or at least 


deliberate. 


3 Agilists argue that if you find yourself part-way through a fail- 
ing project, you can just abort, because “that’s agile.” That’s 
true, and that’s smarter than plodding forward in a sunk-cost 
refusal to face reality, but on a human level it is demoralizing 
and ruins trust to abruptly cancel a project a team has been la- 
boring on for months, “because we’re agile.” Canceling is nec- 
essary, but not free, so you should act as if a Rock is a one-way 
door. 


Use this framework to select the most impactful idea: 
Binstack: Making a maximal multi-dimensional deci- 


sion. This process enshrines “impact” as the highest pri- 


ority, allows other dimensions to participate but neither 
confuse nor dominate the decision, and produces a pithy, 


clear explanation of the decision at the end. 


If youre not coming up with good-enough ideas in the 
first place, try these prompts. But also consider whether 
the real problem is that your strategy is too vague. 


Execs decide, but ideally PMs are in command 

Rocks materially advance the strategy, and executives‘ ul- 
timately own the strategy. So, ultimately the final deci- 
sion of what Rock to execute rests with the executive. In 
practice, however, the PM should be in command of the 
strategy and the ideas, driving the discussion and the de- 
cision. Ideally® the PM is actually in command, and the 
executive is happy to play the role of coach and Devil’s 
Advocate during the decision process, and to sign off on 


a well-thought-out proposal. 


4 At a smaller company, this means whoever has the executive 
role, whoever is running all of Product, or R&D, or the CTO, or 
the CEO. 


5 We won't cover workplace dynamics in this article, so suffice to 
say that this is what “healthy” looks like, and the further reality 
is from ideal, the more one or both parties needs to change. 


Sand maximizes Throughput 


Because Pebbles are the Goldilocks of work-items, it’s in- 
structive to leap over them and solve for the smallest 


items, establishing bookends around the middle-child. 


Ve 
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Duration: <1 sprint 

Sand are items that don’t need to be broken down. 
They’re short, and typically “just need to be done” with- 
out discussion or ado. They can’t take longer than a 


sprint; sometimes they take less than an hour. 


There are a million little things that are individually un- 
measurable but that add up to a significant impact. Great 
user interfaces require a hundred tweaks to attain great- 
ness. High-quality error-handling requires esoteric cor- 
ner-case unit-tests. High performance is often the result 
of innumerable optimizations. High-quality code means 
fixing myriad little bugs, some of which customers never 
experienced. Continuous, low-risk tech-debt reduction re- 
quires a hundred small refactorings. Security patches and 
library upgrades are mandatory maintenance. 
Documentation tweaks are helpful, low-risk, and should 
be done continuously. Great writing in general requires 
myriad tweaks, not grand organizational “pivots;” most 


edits to this document change a single sentence. 


In bulk, Sand is mandatory for wonderful, high-quality 
software. When you use a piece of software and say, 
“Wow, this is really well-done,” that’s a result of Sand. 
While each grain is typically impossible to “measure,” ig- 
noring them means we will never attain greatness, even 
with the best strategy. 


Maximize throughput 

Sand has a material impact only when executed en 
masse, as a sort of opposite to “death by a thousand 
cuts.” Therefore the goal is to maximize how many of 
them we fit into the interstitial spaces between the Rocks 
and Pebbles. The measure of success is throughput: How 


many we complete per sprint. 


We cannot—and should not—try to measure or prioritize 
“fmpact.” It’s too small to measure® It’s enough to agree 
that fixing ten bugs this sprint is a great accomplishment, 
and that our customers and support techs will thank us 


for it. 


6 It is conceivable that two days’ work has a measurable impact 
on some metric. When you happen across one of these four- 
leaf clovers, you obviously should do it, regardless of whether 
you consider it Sand (because it’s fast) or a Pebble (because it 
has impact). The point is to grant yourself the grace for Sand 
to not have a measurable impact. 


Also, high-throughput is fulfilling and energizing for 
teams. It just feels good to cross lots of things off a list. 
Do not discount the importance of the feeling of produc- 
tivity and usefulness. 


Beware: Administrative overhead destroys 
throughput 

Exactly because each grain of Sand takes little time, man- 
agement processes will dramatically bloat the total time 
from conception to prioritization to completion. 
Administration is the biggest impediment to throughput, 
therefore PM’s must ruthlessly fight against the natural 
urge to debate and arrange and carefully assess and esti- 
mate the work, instead of just going about the business 
of completing the work. 


If you’re using the same processes to prioritize and define 
Sand as to define Pebbles or Rocks, the process is wrong. 
Add up the time you debate the merits, craft templated 
user stories, fill out the fields in JIRA, vote and prioritize 
into this sprint then re-prioritize into the next and the 
next, and assign and balance work across people. Don’t 
forget to multiply meeting time by the number of people 
in the meeting. This can easily occupy more time than it 
takes to complete the item in the first place. 


Sand will often originate from engineering; notice how 
many of the examples above are internal requirements 
for great software development rather than customer-dri- 
ven requests. In these cases, it’s often a waste of time to 
perform the usual formalities like user-story-writing, be- 
cause the engineers already know what to do, and why. 
You can observe this waste in senseless force-feeding of 
uninformative “proper product language” and template- 


filling, e.g. 
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Type: Task 
Priority: Moderate 
Source: Engineering 
Estimate: 1h 
Must-do by: 2022-03-15 


Story: As a software developer I want to upgrade 


to 1.2.3 so that I don’t have a security 


vulnerability. 

Linked Goal: Adhere to corporate security policies of ap- 
plying security patches within thirty days 
of a known vulnerability of “low” severity 
and “low” risk. 

Risks: Might break unit tests, might require refac- 
toring around new APIs or behaviors, 
might delay other work scheduled in the 
sprint 

Testing Normal unit tests 
Considerations: 
Acceptance Nothing changes after the upgrade 
Criteria: 


It can get much worse, all for something that could just 
be a one-liner to “upgrade minimist to at least v1.2.3 
because of a security patch,” and in practice the work 
will usually be done in a few minutes, changing one line 


in package.json and re-running the unit-tests. 


Prioritize with intuition and desire, not math and 
metrics 

Because Sand is largely un-measurable, complex prioriti- 
zation systems won't produce meaningful results. 
Fortunately, exactly because they don’t take long to im- 
plement, it’s typically not important when they’re done, 
and indeed many never will get done because we always 
have more ideas than time. 


Therefore, this is a great opportunity to engage people’s 
emotions, and go with things people want to do. Since 
delivered-value is near-nil, why not choose things people 
have energy for? This increases happiness, morale, and 
often quality. Because people naturally work harder and 
better on things they want to do, you get “productivity 
for free,” which in turn increases throughput, which is 


the primary goal. 


Self-managed teams schedule their own Sand 
In keeping with the rule to minimize administrative over- 
head, teams should schedule Sand themselves, not de- 


bate prioritization with executives. 


If the management is constantly engaged in Sand-sched- 
uling, something is amiss and needs to be corrected, and 
it’s always the manager’s fault. Perhaps the team is per- 
fectly capable of scheduling Sand; in that case, the man- 
ager is micro-managing, which is the manager’s fault. 
Perhaps the team really doesn’t understand the customer, 
the market, the technology, or the work, and therefore 
truly is incapable of scheduling Sand; in that case, the 
manager has hired incorrectly or not created an environ- 
ment where the team can understand these things, which 


is the manager’s fault. 


Little requests and suggestions are normal; indeed, these 
will come from all over the organization. But if you’re de- 
bating with spreadsheets and virtual-sticky-note-boards, 


it’s gone too far. 


Pebbles maximize ROI 


Pebbles are not a “balance” between Rocks and Sand; 
they are their own creatures. Their timeframe is defini- 
tionally constrained between that of Sand and Rocks; the 
more important difference is that while Rocks are strate- 
gic, with a view towards winning over the next few 
years, Pebbles are tactical wins that have an impact in 
the next few months, attacking the challenges you're fac- 
ing right now, or a great feature idea you can surprise 


customers with sooner than they expect. 
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a 
se 


a ie 


“We'll start with some small pebbles.” 


Duration: 1-4 sprints 

Pebbles take multiple stories and possibly sprints. Unlike 
Sand, they do need to have a measurable impact; you 
can’t spend a month or two of the team’s time and have 


nothing objective to show for it. 


It is difficult to craft great Pebbles, because impactful 
things have a tendency to explode in effort. Anything 
longer than four sprints invokes the Hofstadter problem 
of time and impact, and therefore must be analyzed and 
prioritized as a Rock. If a Pebble starts expanding, you 
have two choices: 


1. Reduce the scope of the idea so it can be achieved in 
a smaller timeframe, or 
2. Move the item into the “Rocks” category, where it 


will be prioritized appropriately. 


In both cases, you often discover that the impact is no 
longer big enough”. Either this means the team needs to 
get more creative in how to deliver more impact with less 
effort, or maybe this idea simply isn’t a good-enough use 


of your time. 


7 For (1), reducing scope might also reduce impact, in number of 
customers affected or in the magnitude of the effect. For (2) 
the impact might have been great for a one-month project, but 
too small when compared to other large projects. 


Pebbles maximize ROI 

If Rocks maximize strategic impact over the long-term, 
Pebbles maximize immediate impact in the short-term. 
Said another way: They are the “most effective use of the 


team’s precious time.” 


Pebbles maximize ROI: A measure of value, divided by a 
measure of effort, resulting in a metric of efficiency. 
Modest value won in a short time, or more value over 
more time, are both great uses of time. What you cannot 


do is deliver little value but still take a long time. 


Beware the surprisingly high impact of estimation 
error on ROI 

The Hofstadter problem is magnified with ROI calcula- 
tions, so you have to be especially careful, especially with 


classic frameworks like rubrics. 


For example, consider a task that ends up producing 20% 
less impact and ended up taking 50% more time than ex- 


pected—a common real-world result: 


Estimated Actual 


Impact 60 48 
Effort 4 6 
ROI: 15 8 


This item has half the ROI than we originally thought. 
This is an immense magnitude of error, swamping the 
signal you thought you computed. Many other items’ ROI 
will fall within this range of error, telling us that the 
noise from the error exceeds the signal from analysis. 
Which means the rubric is useless. 


To avoid this issue, use this framework for perform- 


ing an ROI analysis. 


Product Managers decide Pebbles 
You could argue that the Product Manager’s most impor- 
tant job is to make decisions exactly like this one: Which 


Pebble will we tackle next? 
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Ultimately it’s a judgement call: A synthesis of what cus- 
tomers need most, what competitors are doing, what is 
consistent with the strategy, what is best for company 
metrics, and what is best for customer delight. Input 
from many directions is appreciated, but one mind needs 
to make the call. That should be the mind closest to the 
customers, to the product, to the competitors, and to the 
market, not a drive-by decision from a manager, and not 
a hostage negotiation with engineers who would rather 


rewrite a whole module from scratch. 


That said, committing a few months of time to anything 
is a big decision, which means it should have a sensible 
justification, and some objective measure of impact so we 
can see whether this activity is having the desired effect. 
With “self-managed teams” comes not only the freedom 
to decide and act, but the responsibility to own the 


results. 


A simple sprint-planning system 


How can you put all this together in practice, in real 


sprint-planning? 


Schedule things in this order, skipping one if there is in- 
sufficient capacity to make significant progress on it giv- 
en the other items in the list, or if high-quality stories 


aren’t ready-to-work: 


1. Time-critical items, regardless of size. (Examples: 
security patches, bugs actively impacting customers, 
critical work for a launch or other event with an 
externally-imposed, immovable date) 

2. One or more stories from the current Rock. 

3. One or more stories from the current Pebble. 

4. Sand. 


Life is never as simple as that, so here’s how to manage 


the common issues: 


“Time-Critical” takes up so much time, we can’t 
make progress on Rocks and Pebbles 

You have a meta-problem: Your team doesn’t have 
enough time to be effective; solving this is now your 
top priority. This problem is even more important than 
your Rock, because in this condition you won’t actual- 
ly complete any Rock. There are myriad causes, and 
maybe multiple simultaneously: Is it a problem of in- 
dividual productivity, of the team owning too many 
things, of architectural dependencies, of the problem- 
domain requiring more people, of lacking specialized 
skill sets, of greater fortitude of saying “no” to certain 
requests, or what? You must diagnose and cure the 


disease. Schedule sprint time to work on the solution. 


Starving the Rock 

Just one story per sprint will cause too much context- 
switching, and take too much calendar-time. If you’re 
constantly starving your most strategic item, this is an 
impediment that the PM needs to address with the 
team. You’re probably falling prey to the Eisenhower 
Matrix fallacy of working on things that are urgent, 
rather than things that are important. Maybe you need 
to pause your Pebble for a while? 


More than one Pebble 

If you’re truly going to execute on all four sections, 
you don’t have time for two Pebbles at once. Plus, the 
context-switching is worse for morale and for produc- 
tivity. The exception is when a Pebble is essentially 
complete, or will be blocked for at least a week, and 
therefore you truly do have the time to work on some- 


thing else. 


Always the Rock, never the Pebble 

Because Sand is definitionally small, a few of those 
items always fit. But, a significant story for a Pebble 
may not. What if Pebble stories never fit? Consider 
that this might not actually be a problem, if the Rock 
is so valuable. Declaring the Rock “the theme of the 
next four months” might be exactly what the team 
needs for focus and maximum impact. It could be that 


once the Rock is “done” (in the sense of “first com- 
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plete version”), you can then tackle a Pebble and then 
another, just doing incremental (small) updates to the 
Rock as you continue to learn and evolve its result, 
rather than immediately tackling an entirely new 
Rock. 


Zero Sand 

Since it’s last on the list, and often aren’t even full sto- 
ries, it’s easy to just never do Sand, but this will result 
in a poor product and unhappy engineers (as much of 
their internal work falls under this category). Consider 
toning down stories from other areas, maybe pausing 
the Rock or Pebble for one sprint, or even having a 
sprint devoted only to Sand, as a fun way to get a ton 
accomplished in a short amount of time and to break 


up the monotony of the sprint-planning cycle. 


Starting a new Rock or Pebble too soon, rather 
than creating more quality and value from the 
ones that just “completed” 

With long lists of genuinely terrific Rocks and Pebbles, 
it’s tempting to start a new one as soon as the current 
one is complete. But software rarely works that way. 
Between learning how customers actually use (or 
don’t use) things in the field, completing small items 
that were originally deferred (so that we shipped 
sooner and started learning sooner and started selling 
the feature sooner), and both incremental and signifi- 


cant follow-on functionality, often you need to keep 


the Rock and Pebble around longer than it first 
seemed, or at least not start a new one quite yet. This 
is realistic for great software and a healthy, sustain- 
able pace of work, and this is another reason why our 
“time estimates” on both Rocks and Pebbles are sub- 
ject to Hofstadter’s Law, and further justifies our dra- 
conian admonitions about identifying and prioritizing 
that work. 


Trying to “balance” every sprint 

It’s not important that every sprint is perfectly bal- 
anced between all types of work. It is important that 
we're balanced over a period several months, other- 
wise something important is getting starved. Indeed, 
it’s often wise to build imbalanced sprints intentional- 
ly, because that means greater focus, less context- 
switching, and therefore getting more quality work 


done. 


Of course all this is easy to say, but hard to execute. Still, 
when we write it down as simply as possible, and try to 
honor it, we'll make better decisions sprint by sprint, 


which in turn creates the most impact year by year. 


It’s tricky to navigate, but this framework will help you 


spend everyone’s time more wisely. 


Printed from: A Smart Bear 
https://longform.asmartbear.com/rocks-pebbles-sand/ 
© 2007-2023 Jason Cohen W @asmartbear 
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JIT selection from independent streams: An alternative to the “big 


backlog” of work 


by Jason Cohen on August 23, 2022 


The vaunted “single-threaded, ordered list” confuses 
“prioritization” with “work-planning,” and forces 
comparisons of the un-comparable. Here’s the solution. 


“Things have gotten a lot easier since I moved 
everything from my to-do list to my it-is-what-it-is list.” 


Scrum teaches us that the “single-threaded, ordered list 
of work” is the correct way to prioritize the work of the 
future. Innumerable articles—including one from this 
very site—tell us that a single, stack-ranked list is the 


simplest way to clarify priority. 


I find, however, that this is unsatisfying at best, and fails 
to achieve sensible prioritization at worst. It also doesn’t 
solve the political challenges of prioritization with 


stakeholders. 


The following system is simple, and has worked for me. 


Prioritization # Work Planning 
Prioritization means: What is most important? 


Critically, it does not mean: What will we work on next? 


Product Managers typically conflate these, trying to do 
both with a single process, and a single backlog. A tradi- 


tional backlog efficiently encodes the answer to the sec- 


ond question, but isn’t the right tool for answering the 


first question. 


You might say, “But we should work on whatever is most 
important, right?” By unpacking the subtle answer, we’ll 
arrive at a better way to prioritize, and a better way to 


plan work. 


Why “the big backlog of work” isn’t good 
for prioritization 


Prioritization 
If prioritization answers “What is most important,” it 
begs another question: What do we mean _ by 


“important?” 


e Important to calm our largest customer who is 
threatening to leave 

¢ Important to catch up to our biggest competitor on a 
specific feature 

e Important to unleash the sales and marketing teams 
onto a new market segment 

e Important to satisfy our security policy 

e Important to decrease the largest cause of support 
tickets 

e Important to mitigate the largest cause of 
cancellations 

e Important to refactor code so that engineers are 
happier and more productive 

e Important to increase profitability 

e Important to achieve our long-term strategy, even if 


there’s no next-quarter revenue 


Sorting a “Big List” of epics doesn’t seem like the right 


way to tackle this complexity. 
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The first problem is that you’re sorting things that are in- 
comparable. Every bullet point above names something 
undeniably “important” but also belies a different moti- 
vation, with different impact, measured differently, so on 
what basis, with what metric, would you order The Big 
List? 


The second problem is that the resulting decision is un- 
satisfying to stakeholders. Sales wants a few specific 
competitive features, and doesn’t have the context to un- 
derstand how those are interleaved with requests from 
support or why the engineers need an entire month to 
“clean up a mess, which by the way, they made them- 
selves?” The Big List doesn’t yield satisfactory 


explanations. 


Work-planning 

The third problem is that The Big List conflates the idea 
of “prioritization”’—what is most important—with “work- 
planning”—which is the order that specific work will be 
executed. While there’s a correlation of course, often 
work-planning does not match our ideal prioritization 


because... 


H 


. Hard deadlines—often externally-imposed— 
requiring work even if something else is nominally 
“more important.” (e.g. zero-day security patch is not 
on our list of prioritized strategic work, but must be 
scheduled immediately. Or: Have to get specific 
functionality shipped for a live customer event). 

2. We might not currently have capacity on the teams 

who are capable of working on a high-priority thing. 

3. We might not currently have the skillsets to execute a 
high-priority thing. 

4. Dependencies—i.e. A is very important; B is not. But 
A cannot be done unless B is done. Therefore, we 
must work on B first, even though A is more 
important. 

5. Unknowns that have to be investigated before work 

can commence, or even be planned. (e.g. spikes or 

proofs-of-concept) 


6. The amount of work, or amount of risk, of executing 
a high-priority thing might mean we shouldn’t tackle 
it yet. 

7. Getting success on some smaller things, while 
working on longer things, is better for throughput, 
better for morale, and is more agile. 

8. Blockers—dependencies on other work, maybe even 
other teams, before we can make progress on a high- 
priority thing. 

9. Starving—sometimes we’ve put off low-priority 
things for so long, they effectively become higher- 
priority. 

10. Holidays and other seasonal things (e.g. kicking off a 
huge new initiative on December 15th is a bad idea, 


even if it’s strategic). 


The solution is to prioritize items in separate, actually- 
comparable streams, and to keep that prioritization effort 


cleanly separate from work-planning. 


Separate prioritization streams 


Don’t cross the streams!" 

“Why?” 

“Tt... would be bad.” 
—Egon Spengler, Ghostbusters 


The stakeholders 


Folks in Sales may not know which items are most impor- 
tant for Support, but they sure as heck have an opinion 
on what would be best for Sales! It’s fun to sit down with 
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sales reps and sales leaders and ask them to generate and 
then stack-rank! the things they believe would be most 
impactful. Tell them they don’t have to worry about other 
departments or other requirements—this time is just for 


them, and this list is just for them. 


1 Wait, that means a single, prioritized list! Yes. Because this is a 
single group, and we’re asking them a single question from the 
bullet-list above (e.g. “what would be best for increasing sales, 
whether because of increased win-rate or increased average 
transaction size”), we don’t have the problem of comparing un- 
like things, and so the “single list” is a good forcing-function to 
generate a clear prioritization. 


The ideas will pour out. Some ideas come from the last 
sale someone lost; that might be insightful or might be 
noise~ Some ideas are critical causes of lost sales; bonus 
points if you can tie them to specific lost-sales; that’s mo- 
tivating for Engineering as much as for Product. Some 
ideas are just a feeling—“I know I could sell the piss out 
of X.” Don’t dismiss those out of hand in your quest to be 
data-driven; multi-billion-dollar startups have been built 


atop experience-informed gut-instinct. 


2 One way to separate the short-term emotional reaction from 
the important trend is to observe what happens in subsequent 
prioritization sessions. When you see the group dramatically 
shifting the priority of items quarter-to-quarter, that indicates 
the group doesn’t have a consistent sense of what would be 
most impactful. Surely some of the ideas are truly great; this 
team just isn’t sure which ones those are. Sounds familiar— 
Product Managers struggle with this challenge all the time! 


By forcing that group to stack-rank their ideas, they’ll ar- 
gue amongst themselves over which are most impactful. 
This creates empathy for what product managers have to 
do all the time—struggle with prioritization in the face of 
multiple goals and imperfect data. It also means they 
can’t make everything “Priority #1,” which in turn sets 
up a healthier future discussion: When, in two months, 
they complain that “you still haven’t made feature X,” 
you can point out that “X” was number seven on their 
list, so they themselves felt that it wasn’t as important; 
you were just listening to their input. (Of course, every- 
one is allowed to change their mind; it’s useful to run 


this process multiple times per year.) 


You’re not promising to do everything they want, in the 
order they want. That’s work-planning, and also this is 
just one list; you’re going to do this with other groups 
too. And this list doesn’t include the most important 
voice of all—the voice of the customer. You are promising 
that you will take this list seriously, because you genuine- 
ly value their input. That’s more than most Sales teams 
get out of Product Managers. 


Now repeat for other groups. 


Sales Engineering has a view unique from Sales, more 
technical and more in tune with technical customers, of- 


ten the users rather than the buyers. 


Support often knows the product better than anyone, and 
with the right tools and processes, has better data about 
which topics are causing the most problems or confusion. 
Remember that this isn’t only about reducing work and 
costs for the Support team—although that’s already a 
great result. It’s about creating a better customer experi- 
ence, because the customer who never needs to contact 
Support to get their work done, is a happier, more suc- 


cessful, more loyal customer. 


3 There are fascinating data supporting the claim that “needing 
Support less” is correlated with “customer loyalty.“ 


Engineering also needs their own list. There’s always 
things to refactor, libraries to replace, spikes for new 
technology to try, replacement of a continuous-integra- 
tion system, test automation we neglected before but 
desperately need now, systems that were adequate last 
year, but now we've scaled out of, and so on. Their stuff 
is about operational excellence and a happy, fewer-bugs, 
higher-productivity environment, not about growth or 
the customer. Therefore, it’s easy for these things to nev- 
er be prioritized, which in the long run creates resent- 
ment and grinds development velocity to a halt. But also, 
engineers—like everyone else—have more ideas than 


time, so they should be forced to stack-rank. 
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So... which of these lists are correct? There is truth in all, 


but none contain the complete picture, like the 2,500- 


year-old Buddhist allegory about the The Blind Men and 


The Elephant: 


Source: J. Himmelfarb (artist G. Renee Guzilas) 


It’s the job of a Product Manager to consider all of these 
inputs and deduce the elephant. Keeping the inputs sepa- 
rate and stack-ranked helps the PM with this daunting 


endeavor. 


The customer 

These are internal groups; what about the customer, the 
star of the show! This is typically the purview of Product 
Managers and Designers and UX Researchers, as opposed 
to internal stakeholders. For simple products, perhaps 
there’s only one prioritized list representing our best idea 
of what’s important to the customer, but products with 
non-overlapping areas of functionality or that serve mul- 
tiple personas might have one list per persona or func- 
tional space. So for example you might have separate 
lists for the needs of small versus large companies, or 


first-time users versus power-users. 


Another common technique is to have a list per step in 
the customer journey, reflecting the fact that it’s impor- 
tant for trial users to find success quickly, and also impor- 


tant for paying customers to continuously see value, and 


also important that the highest-revenue power-users to 
stay and grow. As with stakeholders, the motivation for 
the prioritizing items within each category are not 


comparable. 


There is plenty of prior art on this topic. This is essential- 
ly what is happening with Story Mapping or Opportunity 
Solution Trees, where the customer-journey steps or the 
main opportunities each contain a list of potential work- 
items. Indeed, you could argue that story-mapping is just 
exactly the idea expounded here, extending the concept 
of “independent lists” from only customer-focused topics, 


to topics from Sales, Support, and Engineering. 


WO 
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Release #2 


Time 


Release #3 


Necessity 


Alternate types of list 

If you don’t like separating lists by function or by cus- 
tomer journey, here are other systems that some people 
like to use: 


Kano 

You could also create lists using the Kano Model. Imagine 
one list each for Delight, Performance, and Must-Be. In 
general you don’t want to starve any one of these cate- 
gories, but it’s difficult to prioritize items between cate- 
gories. For example, it’s easy to claim that everything in 
“Must-Be” has to be implemented first; after all it’s called 
“Must-Be” for a reason! There’s no point in bringing on 
new customers who won't be successful. That sounds rea- 
sonable, but it’s simply not true that products should sup- 
ply only basic requirements; indeed it’s often the (unex- 
pectedly) Delightful items that cause a customer to stay 


4 of 6 


JIT selection from independent streams: An alternative to the “big backlog” of work 


despite failings. Having separate lists, none of which 
should be completely starved, helps to combat this 


problem. 


The “Four Lists” 


From Jyoti Bansal, summarized by Adam Thornhill: 


1. Customer engagement. Tuning into user feedback 
and desires. 

2. Sales intelligence. Learning from the field to stay 
abreast of the competition. 

3. Technical debt. The debt engineers must address to 
prevent performance issues. 

4. Vision for the future. Where the product should 


head, expanding the addressable market. 


In all cases, all of the lists are important, in the sense that 
you cannot ignore any one of them forever. But you don’t 
have capacity to move all the needles at once, and the 

items within each list are different-in-kind, and should be 


prioritized separately. 


Crossing the streams for work-planning 


Eventually, human beings need to do work. It makes 
sense to select work from these various streams and—yes 
—place the work in an ordered backlog. When you're ac- 
tually doing work, you want a clear answer to “what’s 


next?” without additional meetings and debate. 


“Maybe we'll never know who was first. 
But I can tell you who's gonna be /ast!” 


By waiting until the last second to compare incomparable 
things, prioritization remains organized and clear. By 
waiting until the last second to shift your mindset from 
“most important” to “planning work,” you maximize clar- 
ity in the process of selecting and scheduling work. 


In software, when you wait until the last possible mo- 
ment to do something, it’s called JIT (Just In Time”), 


hence the titular name of this technique. 


You still have the unenviable job of selecting which items 
from which streams are going to happen next (or “soon,” 
in the case of a roadmap). But that was always going to 
be the case—there’s always more work than time, so 
there’s always a selection process. This technique keeps 
that process more organized than it would have been, 
and it allows you to grapple with work-planning puzzles 
separately from prioritization, which at least compart- 
mentalizes the challenge. Divide-and-conquer is a good 


way to overcome complexity. 


We’re obviously in no danger of 
arriving at consensus.” 
—wWarren Buffet 


5 of 6 


JIT selection from independent streams: An alternative to the “big backlog” of work 


You're selecting from the top ideas at the company, not 
from the 1000 ideas everyone collectively has. It’s a lot 
easier to pick the best from 12 instead of from 1000, and 
in some sense you can’t go wrong—on average all 12 are 


probably decent ideas. And stakeholders will be happy. 


This last point is under-appreciated. This process is more 
explainable after-the-fact. Suppose it turns out Sales’s 
first item is impossible to tackle right now, but you can 
knock out their second item because, by the way, it’s also 
Support’s number four and fits nicely into a multi-month 
customer journey improvement project you have. That’s a 
great message to send back. Or, a message like: We 
knocked out two of your top items recently; we’ve been 
starving the engineering list so we’re going to hit a few of 


those in the next few months, then resurface. 


This process also helps keep yourself honest. You can eas- 


ily report on: 


¢ How much of each kind of work are we doing? 

e What is the age of the top three items on each list? 
(i.e. Are we starving something?) 

¢ How much of this sprint is addressing someone’s “top 
3” thing? 

¢ How much work is related to internal projects versus 
customer-facing? 


It also leads to better ideas: 


e Are there trends that everyone is seeing across all 
functions in the business, that therefore should 
inform our strategy? 

e Can we think of new product ideas that hit multiple 
top-three items simultaneously, and thus are 


especially impactful? 


Don’t forget to schedule rocks, then pebbles, then sand. 
That’s an even more primary principle for work-planning. 
Separate prioritization streams help identify which rocks 
and pebbles should be scheduled in the first place. 


When we differ, Charlie usually 
ends the conversation by saying: 
“Warren, think it over and you'll 
agree with me because you’re 
smart and I’m right.” 

—Warren Buffet on Charlie Munger 


Printed from: A Smart Bear 
https://longform.asmartbear.com/jit-backlogs/ 
© 2007-2023 Jason Cohen ‘W @asmartbear 
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The Iterative-Hypothesis customer development method 


by Jason Cohen on September 17, 2022 


A simple but effective system, used to vet what is now a 
Unicorn, for generating insights about how your potential 
customers think, what they need, and what they'll buy. 


Deep in the forest there’s an 
unexpected clearing that can be 
reached only by someone who has 
lost his way.” 


—Tomas Transtromer 


This is a simple but effective process for building knowl- 
edge through interviewing (potential) customers that I’ve 


employed multiple times in the past 14 years. 


This system led to rejecting some startup ideas, selecting 


the idea of WP Engine, and selecting the right features 
during the early years, which then led to hyper-growth, 
which led to a Unicorn company with three dozen teams 


who do customer development for themselves. 


The goal is to uncover the truth, not to 
sell 


I’ve been the interviewee for many startups doing cus- 
tomer development, and their most common mistake is 
that they spend most of the time selling me on how great 
their idea is. 


AnDeeson) 


“I also managed dozens of employees and oversaw 
international travel arrangements. Plundering was 
actually a very small part of the job.” 


If you—a reasonably intelligent, excited, passionate per- 
son—sit down with someone who meets the criteria of a 
potential customer, and make an hour-long sales pitch 
explaining all the features and benefits, with that person 
wanting to be kind and supportive of this interviewer 
who is as passionate as they are desperate, that person 


will probably say something like, “yeah, that sounds pret- 
ty good.” 


So, what have you accomplished? Nothing. If you don’t 
come away knowing something new and actionable at 
the end of the interview, you’ve wasted your time and 
theirs. 


Listening is being able to be 
changed by the other person.” 
—Alan Alda 


In the Lean Startup method they call this “validating your 
ideas,” so it’s tempting to spend the time convincing the 


person to provide you with validation. Instead, your 
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mindset should be: “What does this person know, that in- 


validates something I thought was true?” 


If you’ve set out to confirm your ideas, not to disconfirm, 
then you will easily see the confirmation and convenient- 
ly miss the disconfirmation, and you will have done 
worse than waste your time—you have convinced your- 


self to believe incorrect assumptions. 


This process is a specific way to achieve that outcome: 


Maximizing genuine learning. 


The Process 


1. Goals: What you're trying to learn 

What is it that you’re trying to learn? If it’s a new product 
at a new company, in a B2B space, where the fundamen- 
tal value-proposition is to solve some specific existing 
customer pain-point, the list should include everything 
standing between the customer’s problem and your po- 


tential product; a list might be something like: 


1. What does the “perfect customer” (PC) look like? 

2. What outcomes does PC need to deliver in a typical 
month? 

(e.g. JIBD framework) 

3. What does PC actually do in a typical day? 

(e.g. tools, workflows, things they love, things they 
dread) 

4. What pain points does PC experience today? 

(i.e. what actually happens, and what pain does the 
customer actually know about?) 

5. How does PC cope with that pain today? 

(i.e. what is your competition, including DIY?) 

6. How much would PC pay to eliminate that pain? 
How is PC able to budget and execute the payment? 
(i.e. what are viable prices and terms?) 

7. What is the triggering moment? What causes PC to 
decide: Today’s the day I’m going to buy something? 
(because no one randomly switches vendors) 

8. What causes PC to resist or fear buying? 

(habits of the present, anxiety of change, risk or cost of 


implementation) 


9. Where does PC go to discover and buy products like 

this? 
(i.e. what are the best distribution channels?) 

10. What specific words does PC use to talk about the 
space; what tacit assumptions does PC have? 
(i.e. how should you talk about the product?) 

11. What ultimate, higher-level goal does PC have? 
(i.e. what outcomes are they expecting as a 


byproduct?) 


Decide on your list of goals first, as they will drive the 


content of your interviews. 


Notice what is not here: Asking the customer what to 


build, or whether would buy some feature. 


This has happened to me dozens of times in my 23-year 
career: Asked a customer “Would you buy if we build 

?” and then say “Yes” and then we build it and 
then they don’t buy. Every seasoned Product Manager 
will regale you with the same story. Here’s where we trot 
out the Henry Ford quote: “If I asked my customers what 
they wanted, they would have said ‘a faster horse.” 
Except sometimes “a faster horse” is a wonderful product 
and a successful startup. And sometimes you should in- 
vent a car. In neither case can you find out by talking to 
customers. 


Instead, what you can learn from talking to customers, is 
what their current life is like, which answers the many 


import questions listed above, leading someday to some- 


thing with Product/Market Fit. 


2. Hypotheses: What you currently think the answers 
are 

It sounds funny to write down the answers to these ques- 
tions ahead of time; after all, the whole point of inter- 
viewing is to empirically discover the answers, not pre- 


sume you already have them! “Learning” and all that. 


The first reason to do this comes from the literature on 
the science of predictions. Specifically, people are much 
better and more objective at seeking the truth when 


they’re forced to record their predictions—for example as 
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formal “bets’—and observe how reality confirms or 


clashes with those bets. We all autonomically retcon our 


beliefs in the presence of new information, or just discard 
the new information (i.e. Confirmation Bias). Writing our 


predictions down helps us avoid these fallacies. 


The second reason to write down hypotheses is that they 


will lead us to great questions for the interview. 


Create hypotheses for each of your goals. At least one per 
goal, but more is fine. It’s also fine to have hypotheses 


that aren’t attached to a goal, but you’re really curious 


anyway. 


Here’s an example list of hypotheses I had about 
WordPress hosting in 2009 before I started WP Engine; 
I’ve added a “mapping” to the goal-list above: 


1. [G1,G4] Bloggers with more than 100,000 page- 
views per month have trouble keeping their blog fast. 

2. [G1,G4] Bloggers with more than 10,000 RSS 
subscribers have traffic bursts that take down their 
site, even if the functions just fine under normal 
conditions. 

3. [G4] All WordPress bloggers worry about getting 
hacked, because it’s common knowledge that blogs 
get hacked constantly. 

4. [G3] Serious bloggers spend at least 3 hours per day 
inside WordPress—whether writing or answering 
comments. 

5. [G3] Serious bloggers spend at least 2 hours per 
week on IT tasks related to hosting. 

6. [G5,G6] Some bloggers spend thousands of dollars 
on consultants to make blogs fast and scalable. 

7. [G5] Some bloggers just live with the problems. 

8. [G1,G6] A blogger with 50,000 page-views per 
month will pay $50/mo if these named problems go 
away. 

9. [G6] Bloggers use personal credit cards to buy 
supporting software for their blog. 

10. [G6] Bloggers need to try software before they’re 
comfortable buying. 


11. [G7] When bloggers get hacked, it’s a traumatic 
moment in which they say “I never want that to 
happen again,” and they’re ready to switch hosting 
providers. 

12. [G8] Scary, unclear, expensive to move all your data; 
what if you get there and it doesn’t work? 

13. [G9] Bloggers read blogs-about-blogging for tips. 

14. [G9] Bloggers trust the advice of WordPress 
consultants. 

15. [G2] Bloggers care about driving RSS subscribers 
more than anything else, because those are repeat 
viewers. 

16. [G2] Serious bloggers publish at least four times per 
week, to stoke page views for advertisements, reader 
interest, and Google search results. 

17. [G10] Bloggers call themselves “bloggers,” not 
writers, authors, content-marketers, etc.. 

18. [G10] Bloggers call their website a “blog,” not a 


“website.” 


Even if you don’t know anything about “blogging in 
2009,” Tll bet this set of theories sounds reasonable to 
you. So reasonable, that maybe you’d presume it’s not 


worth spending lots of time validating them. 


But you’d be wrong. (And so was I. We’re all wrong, at 
the beginning.) After dozens of hours of interviews, I 
found that half of these hypotheses were wrong. The 
ones that were correct still needed to be tuned in detail. 
And equally valuably, I discovered additional attitudes 


and behaviors that weren’t in my original list. 


The greatest enemy of knowledge 
is not ignorance. It’s the illusion 
of knowledge.” 

—Stephen Hawking 


For example: It’s not true that most people are willing to 
pay extra for extra security. It turns out that selling secu- 


rity to bloggers is like selling backup software: If you’ve 
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never had a hard drive failure, it’s unlikely you'll pay 
$30/mo for a backup service. Once you experience that 
devastating event, the first thing you do with your new 
laptop is sign up for any service that promises 100% full 
automatic backup. (Selling security to mid-sized compa- 
nies is different; by then, they’re proactively executing a 


formal policy.) 


Another example: While it is true that bloggers want to 
“try before you buy” for most software (whether “trial” or 
“freemium”), this was not the case with hosting their 
website. It’s such a disruption and technical ordeal to 
move their website to a new vendor, they think of it as 
permanent, not temporary. Therefore, “free trial” is not 
the most compelling offer, whereas “free migrations” is. 
By the way, we tried “free trial” anyway—how can people 
not love a free trial!—but the conversion rate was over 
90%, and when we removed the free trial, signups didn’t 


decrease at all. 


It’s important to list even the most obvious, mundane as- 
sumptions, because you'll be surprised how often you’re 


wrong or they need adjustment, even if you’re an expert 
in the field. 


Put these hypotheses in the first column of a spreadsheet, 


one per row. 


3. Questions: What you ask during the interview 
Generating good questions is the hard part for most peo- 
ple. Armed with your hypotheses, however, it becomes 


easy, if you adhere to the following system. 


Questions are places in your mind 
where answers fit. If you haven’t 
asked the question, the answer 
has nowhere to go. It hits your 
mind and bounces right off: You 
have to want to know.” 


—Clayton Christensen 


Questions are designed to test your hypotheses, and to 
suggest better ones. To achieve this, go to the second col- 
umn in your spreadsheet, and write one question per hy- 
pothesis. Sometimes one question can cover a few hy- 
potheses, if they’re closely related. 


Questions must be open-ended. This is where most 
people go wrong. They'll have a hypothesis like the “se- 
curity’ example above, so they'll ask a question that 
“leads the witness,” because they’re still in “selling” mode 


instead of “discovery” mode: 


Blogs get hacked all the time, and when they do it’s 
devastating, right? Would you like it if your hosting 
company had extra security measures to protect your 
blog? 


Of course everyone will say “yes.” Why not? I’d sound 
dumb if I didn’t agree. That’s why it’s a useless question. 
You didn’t find out what the person actually thinks. And 
therefore you didn’t find out what everyone else will 
think when they look at your advertisement or arrive on 
your home page or review your pricing page. The as- 
sumption baked into your question could be wrong, and 


now you'll never know. 
Instead, write open-ended questions that: 


1. Confirm or negate the hypothesis (the point of the 
exercise) 

2. Do not hint at any one specific answer (seek unbiased 
truth) 

3. Invite the generation a specific answer (uncover the 
correct answer) 

4. Invite more information (seek answers to questions 


you didn’t know to ask) 

On “security,” for example: 
Do you ever think about website security? If so, how do 
you think about that? Do you do anything about it 


today? 


Here’s more examples from the hypotheses above: 
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Bloggers call 
themselves 
“bloggers.” 


Serious bloggers 
publish at least 
four times per 
week. 


Some bloggers 
spend thousands 
of dollars on 
consultants to 
make blogs fast 
and scalable 
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Bad Question 
Do you consider 


yourself a “blogger?” 


Do you publish of- 
ten, so Google ranks 
you high in SEO and 
there’s a lot of sur- 
face area for people 
to find you? 

Would you spend 
$2000 on a consul- 
tant, if it meant your 
blog would become 
much faster and 
more scalable, so 
you rank higher on 
Google search results 
and get more traffic? 


4. Iterate the hypotheses 


Good Question 
When you meet 
someone new, how 
do you explain what 
you do in a few 
sentences? 

How often do you 
publish new con- 
tent? Why at that 
rate—what led you 
to that decision? 


How valuable is the 
speed of your web- 
site? Have you ever 
spent money to im- 
prove it? If so, how 
much, and what did 
you do? Did it 
work? Were you 
happy with that 
investment? 


During the interview, take notes in the spreadsheet, in a 
new column, next to each question (which in turn is next 
to each hypothesis). The conversion might deviate from 
the original point; that’s OK, maybe straying will lead to 


learning new things. 


The most exciting phrase in 
science isn’t “eureka,” but rather, 
“that’s funny.” 


—Isaac Asimov 


If you hear anything surprising, ask follow-up questions. 
Surprise means you’re learning, and since “learning” is 
the whole point of the exercise, you should use “surprise” 
as a signal that you should dig deeper. You can use an old 


interviewing trick: Just say: “Tell me more about that.” 


After each interview, consider what supported or contra- 
dicted your hypotheses. Should you alter some of them? 


Not necessarily, especially after just a few interviews, but 


definitely if you’re seeing a pattern. 


Also create new hypotheses (and associated questions) 
based on new learnings. The more insight you can get 
about your potential customers, the better. Don’t worry 
about whether every hypothesis maps cleanly onto one of 
your goals; just accumulate insight. 


There are two possible 
outcomes: if the result 
confirms the hypothesis, 
then you’ve made a 
measurement. If the result 
is contrary to the 
hypothesis, then you’ve 
made a discovery” 


—Enrico Fermi 


5. Stop when it’s boring 

Do you have all the right answers? Who knows. Probably 
not. But when this method stops producing new informa- 
tion, then you need a new method for making progress. 
That could be a proof-of-concept, a high-fidelity demo, 
an MVP (or actually an SLC), or something else. 


When the surprises stop, that means learning has 


stopped, and that means you should stop the process. 


Ee ANDEesay 


“So, that thing you're here to talk about... 
Talk about that a little.” 
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A typical mistake is to do three interviews, and then stop 
because “I’m not learning anything.” With so little input, 
you might not be genuinely seeking to learn. That’s like a 
marketer saying “I tried two variants of my AdWords ad, 
and none of them are better than my first attempt, so I’m 


not going to try any more variants.” 


It’s also possible there’s nothing to learn because you’re 
fishing in the wrong spot. Maybe there’s no patterns be- 
cause you haven’t found a real pain-point that more than 
a few people have and are willing to pay for. That means 


you need different ideas. 


Here’s more detail on how to determine whether the re- 
sult of customer interviews is telling you “don’t pursue 


this idea.” 


Maximizing your results 


So that’s the whole process. 
Here are more tips. 


Emergent segmentation 

You might notice that customers are segmented. 
Meaning, a certain type of customer tends towards one 
set of answers, while another has a different set. In the 
case of security, for example, when you talk to marketing 
departments at large companies, they do think about se- 
curity, whereas when you talk to independent bloggers, 
they almost never do. 


In this case, it’s useful to make a note of the segments 
you think exist. First, write hypotheses and questions that 
you believe are the determining characteristics of the seg- 
ments. You'll ask these at the top of the call. Then, keep 
separate spreadsheets of hypotheses, one per segment. If 
you're lucky, you'll end up with clarity on the types of 
customers, and what each are like. You might choose to 
target one, some, or all of these; regardless, understand- 


ing the landscape is invaluable. 


Discuss price 

This one is controversial; many intelligent people insist 
that you shouldn’t discuss price in early customer inter- 
views because it unnecessarily conflates financial consid- 
erations with the discovery of customer attitudes, behav- 


ior, and pain-points. 


But in my opinion the price-tag is an essential component 
of the interview, because I believe the price is inextricably 
linked to what the product means to that person, there- 
fore how they think about it and how it affects their life. 
Price also determines the business model of the company, 


so it should not be “figured out later.” 


In early interviews for WP Engine, near the end of the 
call I would float a price of $50/mo for a service that 
made their website faster, more scalable, more secure, 
and came with genuinely good customer service. The re- 
sponses were immediate, emotional, and vehement. One 
group was shocked—shocked—that the price tag would 
be so high; they would never pay even close to that 
amount. Another group said they would only buy the ser- 
vice if it were much more expensive, because otherwise 
they know it couldn’t possibly fulfill its promises. (Those 


groups turned out to be emergent segments; see above.) 


Had I not discussed price, I would never have learned 
about the segmentation, or the expectations behind those 


segments, and thus how to price correctly, and for whom. 


I had another startup concept before WP Engine. Insights 
stemming from pricing discussions was one of the prima- 
ry ways I was able to invalidate that idea, which in turn 


created the space for WP Engine, which is now a unicorn. 


Pricing questions can be open-ended (i.e. “How much 
would you expect to pay for ...”), however I think quot- 
ing a specific price is an acceptable breach of protocol. 
Putting a specific price in front of people elicits a strong, 
visceral response. When someone actually visits your 
pricing page in future, this is also the experience they 
will have—reacting to a specific price. It’s smart to test 


what that experience will be like. 
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You can also tie pricing questions into your other ques- 
tions, to test whether the person really values that topic. 
For example, another way to test the hypothesis that 


bloggers care about security would be: 


Would you pay extra for a security package that really 
worked, or do you not really worry about being singled 
out for getting attacked by a hacker? 


By asking if they'd pay extra, and by almost suggesting 
that they shouldn’t bother, you’re testing whether they re- 
ally ascribe value to the concept. This worked in practice 
—most bloggers initially claimed security was important 
to them, but admitted they wouldn’t pay extra to have 


more of it. 


Expect contradictions 

You're going to get all sorts of contradictory signals. 
People are different. Sometimes because they have differ- 
ent goals, different values, different past experiences, dif- 
ferent roles, different projects, or for no discernible rea- 


son whatsoever. So your data is going to be noisy. 


Some of your hypotheses will end up reflecting the varia- 
tion. You might conclude that some number varies a lot 
rather than staying in a small range, or that there is no 
pattern in people’s opinions about some topic. That’s still 
learning: Knowing what patterns don’t exist prevents you 
from making false assumptions. 


Real patterns will stand out from that noise; that’s your 
fundamental truth, that you can build products and 
strategies around. There might not be much of it. All the 


more reason to highlight it. 


Ask them to explain, step-by-step, how they will use 
it. 

The pattern: You ask a customer if they have the prob- 
lem; they do. You ask whether they'd buy your product to 
solve it; they say yes. Then you build it, and they don’t 


buy. Somehow, your interrogation didn’t work. 


Sales conversions are never 100%, but one technique is 
to ask them to describe, in painstaking detail, exactly 


how they will use the product in their daily life. When 


would they open it up, how does it fit into their work- 
flow, which features do they invoke, how do they move 


the outputs into other systems? 


« 


This works because while they really do mean “yes, I 
think that sounds good,” thinking it through uncovers 
barriers that in fact blocks the sale; it turns out they need 
it to integrate better with something, or they actually 
need a feature you weren’t contemplating, or some other 


hiccup. 


Everyone “knows” what everyone else wants, except 
they don’t 

I don’t think I’ve ever conducted a customer interview 
where the interviewee didn’t switch into “market guru” 
mode. This is where the customer stops talking about her 
own life, her own problems, what features or price tag 
would be acceptable to her, and starts talking on behalf of 
other people. 


“T wouldn’t pay $50/mo, but a lot of people would.” 

“T would pay $50/mo, but most people would expect 
this to be free.” 

“I care about security, but most people are completely 
clueless about that.” 

“Well I use a free tool to do that, but most people 
don’t.” 


Of course it’s coming from a good place—they want to 
help you, they want to explain the “state of the market,” 
they want to leverage their expertise. But they don’t 


know what everyone else wants. 


Neither do you; that’s why you’re doing these interviews. 
And when you see all the crazy, different things people 
think, you realize that it takes tons of interviews to un- 
cover even a modicum of truth. The person you’re inter- 


viewing hasn’t done that, so they don’t know the truth. 


One of the hallmarks of successful companies is that they 
found some untapped aspect of the market and owned it. 
That might be a feature no one had considered, or a tech- 
nology that before now hadn’t been accessible, or a real- 
ization that the human touch trumps everything (Zap- 


pos) or that the human touch doesn’t matter at all 
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(Geico). Even if the person you’re interviewing is a mar- 
ket expert, you’re specifically looking for interesting 


holes and niches that the experts haven’t noticed! 


So be polite, calmly tell them that’s great insight, reinter- 
pret “everyone else wants X” to either mean “I want X” or 
just throw out the comment completely, and redirect the 
conversation back to themselves and their own specific 


situation. 


Leveraging AI... maybe 
AI might help. Perhaps it could... 


¢ Generate hypothesis from the goals, adhering to the 
guidelines, specifying which goal attaches to which 
hypothesis (although of course you'll treat them as 
templates and correct them to what you actually 
believe). 

e Generate questions for the hypothesis, specifying 
which question attaches to which hypothesis, 
adhering to the guidelines above. 

e Clean up transcripts. 

e Scan conversations for common themes. 

¢ Scan conversations for “things that contradict a 
specific set of hypothesis.” 

e Scan multiple conversations for themes. 


However, half the value of this exercise thinking 
through this stuff for yourself. The “aha” moments 


come only when you wrestle with the details. 


You find the contradictions. You discover maybe you 
didn’t think that after all. You realize new ideas that 
solve for the conflicting inputs. You realize which hypoth- 


esis are right, wrong, different. 


So, I advise you not to use Al, except to accelerate busy- 


work such as: 


¢ Clean up transcripts so they occupy less space and 
are easier to process. 
¢ Double-check your thinking after you do the 


thinking, maybe come up with new ideas. 


e Summarize your thinking, i.e. take your thinking that 
you wrote out or said aloud, and make the result 


pithy and clear. 


STFU 


If you’re talking, you’re not learning. To maximize learn- 


ing, minimize talking. 


When you do talk, it should be because you're clarifying 
your understanding about what they just said, digging 
deeper on the current conversation, or opening up a new 


vein of conversation. 
Further reading 


e 12 tips for customer development interviews (by Giff 
Constable, 2012, just as relevant today as then, 
wholly compatible with this process.) 

e 11 Customer Development Anti-Patterns (also by Giff 
Constable, 2013; sometimes it’s easier to list what 
not to do.) 

e Customer Interviews: Get Actionable Insights from 
Every Interview (Comprehensive advice from Teresa 
Torres, also the author of the fantastic book 
Continuous Discovery Habits that also explains what 


to do with the information you get from interviews.) 


“Thanks for coming in. You've given me a lot to forget 
about after you leave.” 


Printed from: A Smart Bear 
https://longform.asmartbear.com/customer-development/ 
© 2007-2024 Jason Cohen ‘W @asmartbear 
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Finding Fulfillment 


by Jason Cohen on December 25, 2022 


What creates a fulfilling existence? Exploration leads to a 
framework I’ve used for years for myself and the people 
around me. | hope it helps you too. 


“Now the twelfth and final thing you can completely 
change right now to be your true self...” 


Many blogs, resulting in book deals, resulting in speaking 
careers, have been forged by taking a swing at a basic 


human question: 
What creates a fulfilling existence? 


Managers desperately ask, forced by the Great 
Resignation to acknowledge that people are more mobile 
than expected when sufficiently (un)motivated! 
Enlightened leaders ask proactively, because intrinsically- 
motivated people do great work that is just as fulfilling 
for themselves as it is productive for the company—capi- 
talism on its best day. Outwardly-impenetrable leaders 
ask it of themselves to conquer burn-out or to decide 
what to do with the balance of life that follows the soci- 


etal definition of “success.” 


1 Your best talent are volunteers—they can always walk across 
the (now virtual) street for more money, because they’re worth 
it, and because their resume has been incrementally improved 
by the experience at your company. You may protest that the 
grass isn’t greener over there, but rather their feelings arise 
from the modern malaise instigated by COVID-imposed isola- 
tion from genuine relationships and the vile so-called online 
“relationships” fueled by toxic politics. You might even be right, 
but it’s too late. 


What follows is my original framework I used to answer 
this question for myself ten years ago when I made the 
decision to step aside as CEO of the (now unicorn) com- 
pany I founded. I still use it myself and recommend it to 


others. 


First, some of the best prior art on the topic. 


Pink’s Motivation: Autonomy, Mastery, 
Purpose 


Daniel Pink famously surmised? that “carrots and sticks” 
are poor motivators for most people. Worse, they're 
demonstrably counter-productive in common real-world 
scenarios. Instead of external motivators, blatantly de- 
signed for the benefit of the organization rather than the 
individual, people prefer—people deserve—to be driven 
by internal motivation, aligned with genuine personal 
fulfillment. 


2 in his book, though you might prefer to watch his extremely 
popular TED talk 


My subjective experience confirms this science-backed in- 
sight. The best people don’t have to settle for anything 
less than personal fulfillment at work, as evidenced by 
the Great Resignation. And anyway it results in an orga- 


nization that we are all proud to build. 


Pink suggests that internal motivation arises under three 
conditions: 
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Autonomy 

People are fulfilled when they decide what to do and 
how to do it. Counter-examples include micro-man- 
agement, inflexible working conditions, and one-way 
command-and-control structures. Positive examples 
include self-managed scrum teams, work-from-any- 
where-and-whenever schedules, and agreeing on the 
goals of the final product rather than dictating the de- 
tails of what that product is and how it must be 
created. 


Mastery 

Great people want the opportunity to do great work. 
They want to be around other people who are doing 
the same. Experts enjoy deploying their expertise; 
novices with vim and aptitude enjoy learning and 


growing. 


Purpose 

As the janitor famously answered in 1962 when presi- 
dent Kennedy asked him what he did for NASA, “I’m 
helping put a man on the moon.” Everyone—not just 
Gen Z—wants to be a part of something bigger than 
themselves. That could be a noble cause, or something 
more incremental but tangible, like genuinely helping 
another human being in their own endeavors, as one 
might do in a world-class customer service 


organization. 


Other work agrees with and extends these ideas? 
However, I believe that to leap from Pink’s original ques- 
tion—What motivates people?—to my question—What is 


fulfilling?—at least one vital component is missing: Joy. 


3 For example, Self-Determination Theory asserts that motivation 
arises from “autonomy, competence, and relatedness”—essen- 
tially the same thing. 


HE 3 FASE: A reason to be alive 


For thousands of years the Japanese have revered those 
who devote their lives to the mastering of a craft, having 
“craftsman spirit” Rk A &’&{ (shokunin kishitsu). The 
samurai and the flower-arranger hold equal value, the 


chef and the janitor hold equal prestige, when each are 
whole-hearted in their endeavors. Furthermore, world- 
class skill is only half of the meaning of being a 
shokunin; you must also play a mindful and intentional 


role in bettering the community. 


It is not only “Mastery” but also “Purpose.” The Japanese 


have venerated 2/3rds of Pink’s trifecta for millennia. 


An equally ancient concept, that received a direct name 
in the 1960s, is 4- X FA 32 (ikigai)—a motivating force 
that gives someone a reason for living: 


. ikigai ... usually means the feeling of accomplish- 
ment and fulfillment that follows when people pursue 
their passions. Activities that generate the feeling of iki- 
gai are not forced on an individual; they are perceived 
as being spontaneous and undertaken willingly, and 
thus are personal and depend on a person’s inner self. 
(Wikipedia) 


This brings color to the idea of intrinsic motivation as 
well as that of extrinsic purpose. Furthermore, it’s not an 


exaggeration to say that this is a reason to go on living at 
all: 


National Geographic reporter Dan Buettner suggested 
ikigai may be one of the reasons for the longevity of 
the people of Okinawa. According to Buettner, 
Okinawans have less desire to retire, as people contin- 
ue to do their favourite job as long as they remain 
healthy. “Moai”, the close-knit friend group, is consid- 
ered an important reason for the people of Okinawa to 
live long. In 2016, [Héctor Garcia and Francesc 
Miralles published] a book based on this concept: 
Ikigai: The Japanese Secret to a Long and Happy Life. 
(Wikipedia) 


We need purpose, whether it’s the Silicon Valley notion of 
“changing the world” or to be a great-grand-parent. To be 
useful, to be needed, even to be wanted, one person to 
another, is already a higher purpose. 


One person at a time, can be all it takes, as in this 
serendipitous Slack exchange I had with WPGraphQL 
founder Jason Bahl: 
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a4 Jason Bahl Nov 4th, 2021 at 2:59 PM 

This is a fun tweet for me: 
https://twitter.com/mosesintech/status/1456340524818894 
850?s=20 


He's a fairly new developer that is now starting a full time job 
working for a company that came into existence because of 
what WPGraphQL empowers them to create. 

— Moses Maximino Cosme II @mosesintech 

3 years ago | decided to become a developer as a career. 

| quit my job, did @LambdaSchool, surfed couches, ate 

on food stamps and soup kitchens. 


Today | accepted an incredible job to work for 
http:/Jambaree.com making @WordPress and @Shopify 
sites with @GatsbyJS. | made it. 

W Twitter Nov 4th, 2021 


&s F100 ¥7 Gs € 


5 replies 
Jason Cohen nN 4 months ago 
“8 wow, cool! (edited 
m1 € 


4 Jason Bahl 4 months ago 

It is indeed. Not going to lie, | got a bit teary eyed. He DM'd 
me to thank me for WPGraphQL because "it's changed his 
life’ & 


1 & 


a4 Jason Bahl 4 months ago 
this is the stuff that motivates me! 
9’ CG 


(reprinted with permission) 


A “higher purpose” might elicit an eye-roll from the skep- 
tic, but it turns out to be useful even if you’re a cold- 


blooded, mercenary, even authoritarian-style leader. 


Sinek’s Fanaticism: Start with why 


Simon Sinek wanted to know why some organizations 
produce fanatics. Not just customers and employees, not 
just people who like and buy the products, but people 
who personally identify with the company, incorporating 
the company’s brand into their personal brand. When 
competitors build something objectively better, they don’t 
switch, because for them it’s not just about the features 


and the price. They'd work for that company if they 


could, and many do. As a result, employee turnover is 
low and productivity is high. Therefore every aspect of 


the business is stronger. How does this arise? 


Sinek sums up the answer: These companies Start with 
“Why.” Meaning: These organizations have clear, simple, 
compelling raison d’étre, a reason for being, something 
they stand for, something they would never contravene 


with their actions, even if it hurts sales or profitability. 


THE GolDEN 
Circle 


ttow/ 
WHAT 
by 
Simon Swek 


It could be a “higher purpose,” like Patagonia’s incontro- 
vertible mission to save the Earth* or SpaceX’s mission to 
make humanity a multi-planetary species, or the Gates 
Foundation project to save millions of lives by eradicating 


malaria. 


4 Beyond the obvious—sustainable practices, an outdoor-wor- 


employees out of jail if arrested while protesting peacefully. 


But it doesn’t have to be grandiose or holy to be motivat- 
ing. Sinek frequently returns to the example of Apple’s 
“Think Different” campaign; this taps the personal brand 
of those who are (or want to be seen as) independently- 
minded, creative, creators, iconoclasts, and artists, in- 
cluding artists embedded in fields like software develop- 
ment and engineering. Or Salesforce with its “1-1-1” or 


WP Engine with its “Engine for Good"—thoughtful, sys- 
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tematic ways of giving back to the communities they are 
a part of (without pretending to single-handedly “change 
the world”). Or Rackspace with their “fanatical support,” 
transforming the traditional status of Customer Service 
from a begrudged cost-center hell-bent on cost-reduction, 
elevating it instead into the honor of serving others, and 
a critical product differentiator in a market that is other- 
wise commoditized. 


When people are financially 
invested, they want a return. 
When people are emotionally 
invested, they want to contribute.” 


—Simon Sinek 


All of these companies were created by and are still run 
by Gen-X-and-older. So, wanting a higher purpose isn’t 
just “a Gen Z thing,” though clearly younger generations 
do, on average, talk about this more, and more conscien- 
tiously. In any case, the freshest talent and the future 
leaders of the world are Gen Z, so even a leader unsym- 
pathetic to “causes” should realize that having a higher 


purpose is more useful than not having one. 


This confirms and clarifies the “Purpose” component of 
the Pink Trifecta. Sinek approached it from an analysis of 
fanaticism and loyalty to organizations, and arrived at 


the same place. 


Cohen’s Circles & Traps: Joy, Skill, Need 


It is possible to be empowered to work how you want 
(Autonomy), to be leveraging your skills and expertise 
(Mastery), and to be proud of your role in a cause (Pur- 
pose / Why), and yet still dislike every day of your exis- 


tence. More than contentment (ikigai), you need Joy. 


Not only is this possible, it is common. There’s the classic 
example of the startup founder who wakes up six years 


into the journey, realizing she’s been surreptitiously 


brought to a boil, burned out, dreading each day, drink- 
ing too much “to turn my brain off so I can sleep” but ac- 


tually because she’s deeply unhappy: 


Each morning for the past couple months, my first 
thought has been “What could today be like if I didn’t 
work here?” I drift off into exploring what it would be 
like to work at WalMart, or the construction site out- 
side, or as a bagger at a grocery store. It seems so stress 
free. This morning, I locked myself in the bathroom 
with the shower running (don’t want wife to know) 
and cried my eyes out. I haven’t cried in many years. It 
felt great, but only for an hour. 

—Anonymous founder 


Eight years ago I created my own framework for under- 
standing how I could avoid this burn-out trap at WP 
Engine? especially when making decisions that are con- 
trary to ego. The ego says “Being the CEO is the best.” 
Silicon Valley says “The founder being the CEO is the 
Only Way®.” So it’s hard to convince the ego to let go of 
being the CEO, as I did at WP Engine, even if being the 


CEO creates unhappiness. 


5 Whereas I did burn out at my previous company Smart Bear. 
Even selling the business didn’t immediately fix the problem, 
although it was the beginning of the answer. 


6 Typically citing Apple, Facebook, and Amazon while ignoring 
counter-examples like Google, LinkedIn, and Intel, although 
there do exist thoughtful arguments that embrace_those 
exceptions. 


Here’s the insight: Not only do you need all three of the 
following components, but any two alone create a specif- 


ic trap: 


4 of 8 


Finding Fulfillment 


Someday I'll make a Venn diagram where “the middle” isn’t the 
answer, but today is not that day. 


Joy 
You love doing it. When you do it all day, you forget to 
eat and pee. At the end of a long day of doing it, you 


still want to do it. 


Skill 

You're great at it. Your work is so good, even you are 
proud of it. It stands out, and others notice. Those 
who don’t understand how much effort you expended 


say: “You’re a natural.” 


Need 
The company needs it done. It’s a top-three priority. 
Doing it well means a critical part of your strategy will 


succeed. Not doing it is crippling. 


Having any two without the third creates a well-defined 
yet common trap. It’s instructive to understand the traps, 


because it can feel good to be in the trap: 


Trap: Joy + Skill — Need = Useless Flow 

At the intersection of Joy and Skill is “being in the 
zone,” a.k.a. Flow”. Wonderful! Unless you’re working 
on something the company doesn’t need done. Being 
in flow is intoxicating, and does “recharge the batter- 
ies,” but it’s unproductive. Continuing the example of 
the burned-out founder living in this trap, there’s no 


one willing or able to tell you to your face that you’re 
not contributing to what the company needs, so you 
stay in your happy place. A classic example is the tech- 
nical founder writing code instead of making sales, 
fixing the website, handling the accounting, or hiring 


the next great team member. 


7 Flow is when a person is fully immersed in an activity for an 
extended period of time. Time passes unwittingly, performance 
is at maximum, and it is universally described as pleasurable. 


Trap: Joy + Need — Skill = Indulgent Failure 

The company needs to begin advertising. You’ve never 
done AdWords before, but you always wanted to try it, 
and anyway it’d be fun to learn something new. You 
might hire someone to do it later, but not now, be- 
cause how do you hire and manage someone else 
without understanding the job yourself? So you make 
the AdWords campaigns. And since you’re unskilled at 
both marketing and AdWords, you waste three 
months. You might even erroneously conclude “Ad- 
Words doesn’t work” because that’s an easier conclu- 
sion than “I don’t know what I’m doing.” If the compa- 
ny needed advertising so badly, you did a disservice to 
the company by indulging your desire to “play with 
it,” because now months have passed without accom- 
plishing what needed to be done. You tell yourself that 
now you can hire that person, because three months of 
flailing somehow makes you an “expert.” As founder, 
no one will contradict you, so you indulge, and the 


company falls that much further behind. 


Trap: Skill + Need — Joy = Burn-out 

This is classic burn-out. When you do the work all day, 
you feel drained and exhausted rather than energized 
(as you would if it were Flow = Skill + Joy). You do 
the work, because the company needs it done. You do 
the work, because you are undeniably great at it. Even 
though you hate doing it, you’d rather take it on your- 
self rather than foist it on others, whether because you 
want to “protect them from the drudgery®” or because 
you believe they can’t do as good a job as you can, or 


because you can’t afford to hire someone. Because you 
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create great results that the company needs, it doesn’t 
look like a problem—not to you, nor your team. But 
because you dislike it, you grow to resent it, and even- 


tually you can’t face it, and you're finished. 


8 Most people don’t want the boss always doing all the drudgery. 
They wouldn’t mind helping. In fact, that can be great for 
bonding. In fact, some people actually like the thing you hate, 
and you're preventing them from being happier. 


I created and used this framework nearly a decade ago, 
to recognize the wisdom of changing roles from the CEO 
of the company I founded (WP Engine) to the CTO, so 
that both our new CEO and I could operate in the center 
of our Venn diagrams. The thousands of people who have 
since worked at our company concur that this was a fan- 
tastic decision, both personally for the two of us, and for 
the success of the business, which is now a unicorn and 


an iconic landmark in Austin, Texas. 


Co-winning E&Y’s Central Texas Entrepreneur of the Year 
Award in 2017 with our inimitable CEO Heather Brunner, who | 
frequently refer to as “the founder who joined three years in.” 
Copying how Reid Hoffman describes Jeff Weiner at LinkedIn, 
it is neither an exaggeration nor unearned appellation. 


This framework is not a replacement of Pink’s model; it is 
compatible. It adds the missing Joy’ component, while 
reinforcing “Mastery” with the label of “Skill.” It lacks 
“Autonomy,” however, perhaps because I created it with 
the founder in mind—a person who definitionally pos- 
sesses autonomy, even to their detriment. “Need” is more 


tactical than “Purpose,” really about being useful. 


Therefore, my recommendation is to identify that higher 
purpose, as described by “Start with Why” or ikigai, and 
fulfill your own part in that purpose at the center of the 


three circles. 


When we intentionally create a work environment where 
others have a chance at fulfillment, we’re already im- 
proving the world substantially. When we care enough 
about others to find out what fulfills them, we can help 
that materialize. When we’re closer to our own center of 


fulfillment, we can better serve others. 


Easier said than done, but do it you must. 


How to find your circles of “Joy” and 
“Skill” 


“Know thyself?” is easier said than done, so here’s some 


help in actually executing on this framework. 


9 Inscribed at the Temple of Apollo at Delphi, independently 
opined by many people throughout history, Thales gave this as 
the answer to the question: What is the most difficult thing? 
When asked what is the easiest thing, he answered: To give ad- 
vice. Which is what I do in these articles. The truth hurts! 


Here’s some prompts to get you started: 


e Even whenI wasakid,I would __, and [still 
find myself drawn to it. 

e LatelyI find thatI love it whenever __, which 
would surprise my ten-years-ago self. 

e If I could go back in time, I would tell myself to 
worry less about. and moreabout 

e When I’m on an extended vacation, I get itchy to 


____3 I just can’t help it. 
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e My parents/friends always laugh when I start talking 
about _ because I get so excited I can’t stop 
talking about it. 

e My parents always saidI would bea___ because 
even when I was three years old... 

e Whenever! __, I get lost in the work, and feel 
energized (not exhausted!) when the work ends. 

e If I could go (back to) college, I would get a degree 


in 


¢ The last project I really enjoyed, and would totally go 
back and keep working on, was 

¢ I was surprised how much my peers praised my work 
whenI___; maybe I’m better at that than I 
thought. 

¢ Recently I was totally immersed, engaged, excited, 
and happy while doing 

¢ I asked a few people who know me, and who I trust 
to be thoughtful and observant, and they said my 


special strengths are 


Just as useful are anti-examples. Sometimes it’s easier to 
avoid something you hate, then to construct a situation 


where you're constantly doing something you love: 


e I have an intense dread of any meeting where we 


e The lasttimeIhadto _—_—s,Ididan 
embarrassingly minimal job, because I couldn’t bring 
myself to do better. 

¢ If my job starts requiring meto__—_—swiwith even 
10% of my time, I would at least think about 
changing jobs. 

e WhenI’m faced with _I physically feel the “pit 
of my stomach” falling. 

e WhenIm faced with _I procrastinate even to 
do chores that I dislike and would normally avoid. 

e If I’m being honest, although I would really love to 
be great at , the fact is ’m just never going to 
be good at it. 

e Whenever! _all day long, I know I do great 
work, but I’m absolutely exhausted; the rest of the 


night will be vegging out of the sofa with mindless 


entertainment. 


e Iknow!m supposed tolike/do__—_, but the 


truth is I can never get excited about it. 


You should be honest, even if you think “society” 
wouldn’t like your answers. There’s nothing wrong with 
the drive to make money, or to become famous, or to 


prove a point. 


Desire to seem clever, to be talked about, to be remem- 
bered after death, to get your own back on the grown- 
ups who snubbed you in childhood, etc., etc. It is hum- 
bug to pretend this is not a motive, and a strong one. 
Writers share this characteristic with scientists, artists, 
politicians, lawyers, soldiers, successful businessmen— 
in short, with the whole top crust of humanity. 
—George Orwell, Why I Write 


The best scenario is when you are doing your calling. As 
Tim Ferris describes it, when anyone asks him whether 


they ought to write a book: 


If the rewards are 1/4 of what you think they will be, 
and it takes 2x longer than you think it will take, is it 
still a no-brainer to do it? Not just “good” but a “no- 
brainer,” like you almost have to do it? 


A final note on “Skill”—there is the skill you already 
have, but there is also the skill you aspire to have, that 


you're excited to develop. 


Iam always doing that which I 
can not do, in order that I may 
learn how to do it.” 


—Vincent van Gogh 


This is useful to highlight, because learning is often ful- 
filling. The healthiest work environment is where your 
existing skills help you succeed at the nominal job de- 
scription (so youre operating from a position of 
strength), and where the job permits or even encourages 
your learning in the direction you want to develop (in- 
curring the costs of learning, such as work taking longer 


than it ought to and making avoidable mistakes). 
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Finding Fulfillment 


; : | 
I wish you luck on your journey! Thank you to Jason Evanish for providing feedback on earlier drafts. 


Printed from: A Smart Bear 
https://longform.asmartbear.com/fulfillment/ 
© 2007-2024 Jason Cohen yy @asmartbear 
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Navigating the unpredictability of everything 


by Jason Cohen on March 19, 2023 


We dramatically, repeatedly fail to predict the future. Does 
that mean “strategy” is senseless? No, it means you need 
these techniques to navigate a volatile world. 


Predicting markets 


Analysts at Goldman Sachs spend their whole life learn- 
ing advanced mathematics, building sophisticated models 
of complex corners of the world, and are financially in- 
centivized to predict the future accurately, because 
Goldman Sachs makes billions of dollars if they can be 
10% more correct than the next firm (who also employs 


brilliant analysts). 


So, how accurately did they predict economic metrics 
within their area of expertise? They got it very wrong, for 


25 years, often not even directionally correct: 


| —— — 3-Month T-Bill Rate 
9% Board Estimated T-Bill Forwards Curves 
T-Bill Forwards Curves (31 Dec 2018) 


1987 1990 1993 1996 1999 2002 2005 2008 2011 2014 2017 2020 2023 


Thick blue line is the actual value of the metric; grey lines are 
quarterly predictions of how the blue line will move, from each 
starting-point. 


Here’s another: 


Chart 4.1: Oil and gas receipts: outturns and forecasts 


16 Peet March 2015 EFO™ 


—— Outturn 


ee rn asa goiter nent 


12 —— Successive OBR o/s 


LE pp ig i ig i i i 
2000-01 2002-03 2004-05 2006-07 2008-09 2010-11 2012-13 2014-15 2016-17 2018-19 
Source: HMRC, HMT, OBR 


But maybe the macro economy is too hard to predict. It is 
“chaotic,” we are told, in the mathematical sense that 
small changes in inputs result in large, unpredictable 


changes in outputs. The “Butterfly Effect.” 


ne 


“So you believe that when you flap your wings on one 
side of the planet, all kinds of crazy things are caused on 
the other side? That sounds like a lot of responsibility.” 


When we create strategies, we’re told to “skate where the 
puck is going.” That means predicting the future, such as 
macro economic trends, and the trends of our industry 


and immediate competitive markets. 


But if the smartest, most motivated experts can’t do that, 


why do we think we can? 


Let’s try something closer to what corporate strategists 
must do: Predict the future for products in well-under- 


stood industries. 
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Predicting sales 


Researchers at McKinsey & Company, a consulting firm, 
studied the sales forecasts for new drugs made by ana- 
lysts at brokerage firms. The data included more than 
1,700 individual forecasts on 260 drugs over a recent 
ten-year span. ... The average error was large. Almost 
two-thirds of the estimates missed the peak revenue 
amount by 40 percent or more. Further, forecasts for 
follow-on drugs were no better than the first launch- 
es within a therapeutic class. 

—Michael Mauboussin & Dan Callahan in Total 
Addressable Market, Credit Suisse 2015 (my emphasis) 


Once again, highly incentivized, highly trained analysts, 
within their area of expertise, within a well-understood, 
highly-regulated industry, mostly get it really wrong, 
whether predicting sales of a new drug, or predicting 


sales of a new competitor to an existing drug. 


Mauboussin and Callahan suggest that one way to in- 
crease the accuracy of predictions is to “use base rates as 
a reality check.” That is, use the industry average as the 


starting point for your prediction. 
Does this method work? They illustrate: 


Here’s an example of how the base rate approach can 
figure into your judgment of TAM. During a conference 
call in February 2015, Elon Musk, the chairman and 
chief executive officer (CEO) of Tesla Motors, suggest- 
ed the company might be able to achieve a 50 percent 
compound annual growth rate of sales for the next 
decade ... 


... The base rate method ... simply asks: “What hap- 
pened to other companies when they were in a compa- 
rable position?” Exhibit 12 shows the distribution of 
10-year sales growth rates for more than 1,200 in- 
stances of companies of a similar size as Tesla is 
now, measured by sales. The average growth rate, ad- 
justed for inflation, is less than three percent, with a 
standard deviation below eight percent. Further, no 
companies achieved a compound annual growth 
rate (CAGR) in excess of 40 percent ... We have 
placed a star at the growth rate Musk mentions. 
—Michael Mauboussin & Dan Callahan in Total 
Addressable Market, Credit Suisse 2015 (my emphasis) 


As we see from Exhibit 12, Musk’s call is completely out- 


rageous when using the base-rate method: 


Frequency (Percent) 
NO 
} 
Pd 


<(25) 
(25)-(20) 
(20)-(15) 
(5)-0 

0-5 

5-10 
10-15 
>50 


(15)-(10) 
(10)-©) 
15-20 
20-25 
25-30 
30-35 
35-40 
40-45 
45-50 


CAGR (Percent) 


Exhibit 12: 10-year growth rates for companies with $6B-$13B 
of sales. 
Source: Michael J. Mauboussin and Dan Callahan, “The Base 
Rate Book— Sales Growth,” Credit Suisse Global Financial 
Strategies, May 4, 2015. 


It is 2023 as of this writing, so we can evaluate Musk’s 


call and the base-rate method: 


Tesla’s Race Towards 
Profitability 


Yearly net income and revenue of Tesla 
(in billion U.S. dollars) 


HM Netincome #® Revenue 181.5 | 
80 


60 . 


40 


. allt 


2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 


Source: Tesla 


Statista % 


Actual Tesla revenue CAGR since 2015 is 55% —even more 
than Musk’s 2015 claim. 


So, the base-rate method is suspicious, and the analysts 


are suspicious. Is anything not suspicious? 
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When we build strategies, we’re predicting what sorts of 
products are going to sell, often inside a fast-changing 
market, selling to customers with fickle desires in an 
ever-changing competitive landscape. Drugs that address 
a well-known disease with well-known rates of incidence 
and trends, with well-known competition, are easy in 
comparison. Easy, but already almost impossible even by 
the experts. And using base-rates doesn’t give us much 


confidence either. 


So why do we think we can do it? 


ANC 


Daily Mail, from the year 2000. 

Choice excerpts: “e-mail [is] far from replacing other forms of 
communication” // “the future of online shopping is limited” // 
“teenagers’ use of the Internet has declined ... they’ve been 
through all that and then realized there is more to life in the real 
world and gone back to it.” // And who generate these non- 
sense predictions? “Experts from the Virtual Society Project” 
comprised of “research from 25 universities across Europe and 
the US.” 


Predicting winners 


Chess Grandmasters spend far more than 10,000 hours 
reaching an elite, unbelievable level of skill. They instant- 
ly pattern-match board positions against thousands of 
games they've memorized. They have an intuitive sense 
of how the future might unfold, without actually thinking 
through every possibility. They understand the relative 
strength of the best players, because they’ve studied their 
games for years. So when they guess who is going to win 


the 85th annual Tata Steel tournament—arguably the 


most prestigious tournament on the annual circuit—it is 
more than just “sports betting.” It’s betting by deeply ed- 
ucated experts. How did their predictions go? 


2023 Tata Steel Masters Predictions 
After Round 12 


Probability of Winning by Round | Pawnalyze.com 


100 


Praggnanandhaa 
Van Foreest 
Keymer 

Erigaisi Arjun 
Abdusattorov 
Maghsoodloo 
Gukesh D 
Aronian 


80-5 


60-5 


Win % 


40+ 


Ding Liren 
Carisen 


Png ® R R R R R R R R R 
“yp a 7 a 5 0g Ug hag Mh 5 Ya g Ma he Ma, Ya , 
{7 £4 2 


Notice Giri—initially tiny, then growing, then shrinking by round 
8, then growing again, then being crushed by Abdusattorov on 
round 12. 

Giri won the tournament on round 13. 


The “base rate” prediction is that Magnus Carlsen—the 
world #1 in all three major categories of chess, arguably 
the best player to have ever lived—should win, and in- 
deed he was given the highest probability of winning be- 
fore the tournament began, even up to round 4. 


But the world doesn’t necessarily unfold around the base- 
rate. Nordirbek Abdusattorov—an 18-year-old junior 
player rated more than 100 Elo points lower than Carlsen 
(an enormous gap)—was given such a small sliver of a 
chance at the start that it’s hard to even read the number. 
He lead the tournament standings until the very last 


game. He also beat Carlsen in round 5. 


And Giri was never given more than a 30% chance; usu- 


ally 10% or less. He won the entire thing. 


The same thing happened during the last FIFA World 
Cup. Argentina—the eventual winner—was never given 
much of a chance; even in the quarter-finals the bets 
were on France: 
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ee ——— Probabilities of final victory according to the bookmakers during the Word Cup 2022 


When we build strategies, we’re predicting the competi- 
tive market and which products will be winners. Chess 
and soccer games have a higher variance than compa- 
nies, but analyzing sales funnels sometimes feels a bit 
like predicting Tata Steel. How do you build a strategy 


around this level of unpredictability? 


Predicting product 


Stewart Butterfield always wanted to make a game. So, 
in 2002, he did: 


Refresh | Report a Bug | Save & Exit 


| GAME CO NEVERENDING 


Prozacton 
Your shrink probably has offices here. 


c f.) 
pixel a. shun: Milkshakes in civic center, too 


yeoz: to whomever is eating all the pizza: 
please share :( 


clearing 


Inventory 
Load: 2.58/300 [drop all] 4 
@ jar of rejuvenating sparkle powder [1] 
Fierov ? (B J ptarmigan jerky (1] 
Heights || (B) @ pretzel (1) 
| bottle of water [1] 
® check box [1] 
(B) & scroll bar [1] 
sex submit button [1] 
sheet of paper [2] 
B) @ red sheet of paper [2] 


¢ Center San Poshio Soso Prozacton 


e e e 
Bentown —Lunkhed ? 
Commons 


Game Neverending: An in-browser multi-player on-line game 
“with no way to win, nor any definition of success.” (Like some 
companies | know) 


It failed, but it had some interesting features. Alpha 
testers liked that they could share game objects by drag- 
ging them into chat windows. So, the engineers created 
an online application for real-time chat with image- 
sharing. 


Stewart Butterfield is the founder of Slack, so you might 


guess that this “better chat” idea became Slack, but no. 


The chat application only worked in real-time—your pic- 
tures didn’t stick around when you closed the app. And 
this was fatal because it turns out people were interested 
in the sharing part more than the real-time part. So in yet 
another upheaval they rewrote the Flash application as a 
regular website, and Flickr was born. By 2008 it had be- 
come the largest photo-sharing site in the world with 3 
billion photos and 5,000 more uploaded every minute. 


Yahoo bought Flickr for $25M in 2005; Butterfield left 
three years later. Now that he had money in his pocket, 
he was free to go back and do what he always wanted to 
do, which was to make a game. So, in 2009, he did: 


There’s a whole story about their dreams of a massively- 
multi-player experience, with APIs so others would build 
even more things on top of the game. 


But it doesn’t matter, because once again it was a failure, 
and shut down. But once again there was a piece of the 
game that people really liked, and once again it was the 
chat system. Butterfield pivoted the company with a new 
mission, and a new name that belied the mission: SLACK, 
the Searchable Log of All Conversation and Knowledge. 
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Still not “chat,” but rather a “searchable log of knowl- 
edge.” It’s too hard, Butterfield insisted, for corporate 
denizens to search Google docs, and email, and chat ses- 
sions, and intranets, and knowledge bases, and support 
channels, and sales logs. Slack brings all that content to- 
gether into a single omni-search, thus solving a knowl- 


edge-management problem common to all companies. 


Butterfield was certain that merely “building yet another 
chat system” was a bad strategy, whereas “transforming 


communications” was a good strategy: 


We are unlikely to be able to sell “a group chat system” 
very well: there are just not enough people shopping 
for group chat. 


That’s why what we're selling is organizational trans- 
formation. 
—Stewart Butterfield, Slack founder, in 2014 


Once again, great theory, great mission, great strategy... 
and it didn’t work. Slack was, and is, yet another chat 


system. 


Fortunately for him, Butterfield was also wrong that “just 
chat” wouldn’t sell very well. Slack was one of the 


fastest-growing companies in history: 


YEARS FROM $IM TO $100M ARR 


(Pornerstone 


4 slack servicenuw () shopify ¥,scoupa 
$90 ®twilio 


M4 BLACKLINE 


UN intercom 


$ ARR (in M) 
= 
3 


What it is about people not wanting to build chat? 
WhatsApp has a parallel story. Initially it was just a way 
to post a public status message, so friends could see what 
everyone is up to, hence “WhatsApp” (like “What’s Up”). 
No one cared. Then the iPhone launched push-notifica- 


tions, so they added a feature to alert you when a friend’s 


status changed. People started abusing this as a crude 
form of group-chat. So they added group-chat as a prop- 
er feature. The correct strategy was simple: SMS costs 
money for most people in the world, so a winning prod- 
uct strategy is “SMS, but free.” That strategy worked: 


ED IT (Ghewsflicks To download our visual news app, just sms NF to 52424 
WhatsApp is killing SMS _ 
30 
A 4-year-old company, — Messages sent per day, bn WhatsApp 
is about to triumph % 
over a nearly 20-year 
een a — 7) 


one on service-yet 
another instance of b 
innovation unleashing 


creative destruction 0 
in technology industry Sis & 


6 0SC*t« 05 0 5 


Source: The Economist 


When we build strategies, we’re predicting how cus- 
tomers will receive our product, how it will solve prob- 
lems or be delightful. And yet so often the idea we start 
with isn’t the idea that wins. Even once Slack became 
“Slack” and not “Glitch,” the strategy was still wrong. 
How can we ever be right? 


The Solution 


The Slack and WhatsApp stories of unpredictable success 
have at least two things in common: 


1. They had a strategy—a philosophy of what would be 
entertaining or useful, a design sense of what would 
be delightful, brand new ideas for features and user 
interactions that people loved. They didn’t just throw 
random things at the wall. 

2. They went where the customers took them—if not a 
game, then sharing images; if not another game, 
then better chat; if not corporate omni-search, then 
chat with APIs, if not status updates, then free SMS. 


A strategy is required, even when it’s wrong. 

A strategy gives you a direction. A direction creates 
something interesting. Something interesting might be 
used by zealous early fans, even if it ends up uncovering 
the answer rather than being the answer. 
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IT COULD BE THAT THE PURPOSE OF YOuR LIFE Is 
ONLY TO SERVE AS A WARNING TO OTHERS. 


If Game Neverending wasn’t delightful enough to attract 
alpha testers, there would never have been a reboot into 
Flickr; if Glitch didn’t have 150,000 users at its peak, 
there wouldn’t be the idea for Slack; if Slack didn’t have 
enormous ambitions, it would not have received VC 
funding (for the same reason Butterfield himself gave for 


why “yet another chat” isn’t valuable). 


A corollary: Making a decision and moving forward is of- 
ten more effective than extended deliberation about the 
decision. Deliberation assumes we know how to reason 
about the future, but even experts aren’t good at that. 
Making decisions and gaining experience is how to find 
the right answers, if the organization is introspective 
enough to also face the truth when it turns out the origi- 


nal strategy is incorrect. 


Dispense with the idea that there is One True 
Solution to the puzzles, and that the way to get there is 
to gather enough information. While “iterate fast, learn 
fast” is good advice, it still doesn’t mean you're iterating 
towards the One True Solution. It means you’re taking a 
path towards something, which will turn out to be an in- 
tegration between your own creation and the swirling re- 


ality outside of you. 


More than any other time 
in history, mankind faces a 
crossroads. One path leads 
to despair and utter 
hopelessness. The other, to 
total extinction. Let us pray 
we have the wisdom to 
choose correctly.” 

—Woody Allen 


The customer (behavior) is always (directionally) 
right. 

There is a contradiction in these two points. Point (1) is 
to have a firm strategy, yet point (2) is to go wherever 
the customers are, even if that violates (1). If you just do 


“whatever customers want,” how is that a strategy? 


The answer is: Customers are where you discover how to 
upgrade your strategy. Since you know your initial strate- 
gy is wrong, following customers’ lead is how you correct 
it. 


When a customer loves one feature especially, or asks for 
adjacent features, those requests themselves are not your 
new strategy; the intent behind them might be. So “give 
me push notifications for my status updates” is a feature 
request, but the reason they want that is to circumvent 


SMS, and incorporating that idea is the strategy upgrade. 


Perhaps the most interesting signal is when the cus- 
tomer abuses your product to accomplish something 
else. The desire for that “something else” is so great, 
they’re willing to use the wrong tool to get it done. Not 
only exemplified by WhatsApp users abusing “my status” 
to be “group chat,” but also in my own experience getting 


Smart Bear to start growing: 


Our first product (Code Historian) let you visualize the 
history of your codebase, which seemed cool to me but 


few people paid for it: 
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i [Code Historian] C:\proj\httpd-2.0\server\util_md5.c 
File Options Help 
Eiehieem| Recent Changes y] in C:\proj\httpd-2.0\server\util mdS.c 


HB 11 months 17 months 23 months 23 months 35 months 


#include “ap_config.h” 
#include “apr_porteble.h" 
#include “apr_strings.h” 
#include “httpd.h” 
#include “util_md5.h” 
#@include "util_ebedic.h” 


JAP_DECLARE(char *) ap_md5 binary(apr_pool_t *p, const unsigned char *bu 
{ 


const char *hex = "0123456789abcdet"; 
apr_mdS_ctx_t my_md5; 
unsigned char hash[MD5_DIGESTSIZE]; 


char *r, result(33]; /* (MD5 DIGESTSIZE * 2) +1 */ 
int i; 


yt 
* Take the MDS hash of the string argument. 
"7 


7 apr_mdS_init («my_md5); 
#if APR_CHARSET_EBCDIC 
apr_mdS_set_xlate(smy_md5, ap_hdrs_to_ascii); 
gendif 
apr_mdS_update(smy_ md5, buf, (unsigned int) length); 
apr_mdS final(hash, «my_md5); 


for (i = 0, r = result; i < MDS _DIGESTSIZE; i++) { 
*r++ = hex[hash[i] >> 4]; 
*r++ = hex[hash[i] « OxF]; 


) 
tr= '\0'; 
return apr pstrndup(p, result, MDS DIGESTSIZE*2) ; 
) Author: daugm 
prefix ibapr functions 
and 


2 
|AP_DECLARE(char *) ap_mdS(apr_pool_t *p, const unsigned char *string) with ay 


Fal 


| realize how old this screenshot looks, but it also means these 
insights have withstood the test of time. 


But people abused it to enable peer review. We were get- 
ting feature requests like: “Let me package up what’s on 
my screen and send it to someone else,” and “Let me 


write directly on a line of code and send it back.” 


} 


Line 249: if ( ! foundInLocalList ) 

BD: Is this really sufficient? From what I 
understand, localCheckouts is null only when { 
the changelist is not pending. 

: I thought so too, but it turns out 
localCheckouts can also be null if there are 
no files attached to it at all, so we can get 
here with that. 


// in particular. Cases 71 
IScmLocalCheckout[] localCheckouts = changelis 
|_localCheckouts. 


nd on, 
being non-null (but empty) 
with pending changelists without files! 


System. err.println(); 
System.err.println( "ERROR: Changelist * + 


Elena | Create Defect 
Mark as Read 


return false; 
Comment: | don't think so; I checked all the 


references to getLocalCheckouts } 
(Q and everyone seems to check 
For null. 


Submit Comment 


Currently the only other explaination is a 
System.err.println(}; 
System.err.println( "ERROR: Pending changelist 


Twenty years later, this is a standard feature of many developer 
tools, but at the time it was an innovation that fueled a multi- 
million-dollar bootstrapped company. 


A new product was born (Code Collaborator) which 
within a few years represented 97% of the revenue of the 
company. “Abuse” is a strong, positive signal for customer 


demand, and worthy of a change in strategy. 


So yes, the future is unpredictable. But that doesn’t ab- 
solve you from creating a strategy, and it certainly 
doesn’t mean you should accept that life is a random 
walk, with no way to bias results in your favor. In fact, 


sometimes strategies are correct right from the start, like 
it was with Google, LinkedIn, Amazon, Apple, and my 
latest company WP Engine x 


c Although one reason WP Engine was correct at launch was be- 
cause I used a system to discover what was important before 


building it. 


You need a strategy, a fixation like Butterfield has for a 
certain type of game, a galvanizing reason for everyone 
to act together with a common purpose. And yet the 
strategy is always under suspicion, always updating, al- 


ways reacting to unpredictable realities. 


How, exactly, do you do this? 


Strategies that defeat unpredictability 


While it might not be possible to predict the future, it is 
possible for a strategy to side-step unpredictability 
through these mechanisms: 


Build a moat 

Covered in this companion article, a “moat” is a long- 
term, durable competitive advantage. A structural advan- 
tage that others cannot disrupt, lasting for years, is re- 


silient to the volatility of your competitive market. 


Have more than one way to succeed 

If multiple things all have to go right for a plan to suc- 
ceed, it probably won’t succeed. That sort of plan tacitly 
assumes we can predict the future of many different com- 


ponents; surely a bad bet. 


This is why startups are difficult in the first place: We 
have to have an insight and build a compelling product 
and be able to reach target customers and do that cost-ef- 
fectively and at a price customers will accept and be bet- 
ter than the competition for some segment and not lose a 
key team member early on and not have global economic 


failure. Like this. 


A resilient plan is built of “or” not “and:” We could reach 
customers through social media or paid advertisement or 
influencer marketing or channel sales. A product that at 
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least could be sold in all those ways is more likely to suc- 


ceed than a product which can only be sold a single way. 
Optionality defeats unpredictability: 


e A product that is very low cost to create has many 
options for pricing, and therefore more likely to find 
effective pricing. 

e A product in a large, growing market has many 
niches and personas and channels to potentially 
target, and therefore it’s more likely you'll find some 
combination that works. 

¢ Some indie developers build multiple small products, 
then pour their effort into whichever one happens to 
take off. 


More examples are given in Nassim Taleb’s concept of 


Antifragility. In his words: 


If you “have optionality,” you don’t have much need for 
what is commonly called intelligence, knowledge, in- 
sight, skills, and these complicated things that take 
place in our brain cells. For you don’t have to be right 
that often. All you need is the wisdom to not do unin- 
telligent things to hurt yourself (some acts of omission) 
and recognize favorable outcomes when they occur. 
—Nassim Taleb, Antifragile: Things That Gain from 
Disorder (2014) 


Leverage your assets 

“Strike where you are strong and the enemy is weak” 
said Sun Tzu 2,500 years ago, echoing into strategy-de- 
sign from the SWOT of the 1970s to modern agile frame- 


works like the “Means” in Effectuation. 


This is obvious but often forgotten by entrepreneurs who 
believe they have found lucrative opportunities with 
amazing product ideas, but which fall outside of their 
sphere of competency, where the enemy is strong and 
they are weak. Any venture is likely to fail for many—un- 
predictable—reasons; the very least you can do, is lever- 
age your capabilities, knowledge, network, (professional) 
friends, and accumulated assets. That might mean build- 
ing a very different kind of company or product or target 
market; in fact that’s exactly the point. When you pick 


the battle you are best-suited to win, you have a higher 
chance of winning regardless of the unpredictability of 


the world around you. 


Intentionally reactive 

If you cannot predict the future, one option is to spend 
no time whatsoever trying to plan. Just react to what you 
see now, and use your judgement and a long-term strate- 
gy to solve the immediate opportunities while advancing 


a long-term agenda. 


We know, we know... iterate fast, learn fast, get to the 
right answer fast. We know that, we say that, but then 
we take four months to deliver one feature, or take 18 
months before we’re “ready” to launch. Truly embracing 
and living the idea of constant delivery, constant feed- 
back, constant learning loops, constant adjustment of hy- 
pothesis, quick decision-making, quick updating of prior 
decisions, no ego tied up with who was right or wrong 
about what, treating all doors as two-way doors, more 
than lip-service but encoded into the DNA of the organi- 
zation, can overcome unpredictable barriers. There was 
no real prediction to begin with, just a constant stream of 
hypotheses, continuously updated, possibly using frame- 
works and processes specifically designed for this 


mindset 


2 e.g. building in public; the OQODA loop; Iterative Hypothesis 
Testing; Continuous Discovery, Continuous Delivery (not to be 
confused with Continuous Deployment, in which deployment is 
automated but isn’t happening many times per day) 


Hedged bets 

If you buy 30-year life insurance and live for 60 more 
years, you will make far more money than the insurance 
costs, so you come out ahead. If you buy 30-year life in- 
surance and die in 2 years, your family will receive far 
more money than the insurance costs, so you come out 
ahead. Thus “insurance” is a way to always come out 


ahead, without predicting the future. 


The catch: You’ve reduced your maximum upside. In the 
60-year case, you paid insurance premiums for 30 years 


while receiving no money in return. On balance, howev- 
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er, this “tax” is worth it, because you’re trading slightly 
less maximum upside for predictable, net-positive 


outcomes. 


ee 
y . \ 


“You huff and puff, we move in with our brother, 
and we all split the insurance money.” 


Strategies can also “buy insurance,” i.e. come out ahead 


either way, albeit with a tax. Examples: 


* Multiple vendors for same service® Shift workloads 
based on price and performance. (How would your 
price-negotiations go, if you were in this position of 
power?) 

e¢ Multiple brands (whether created or acquired). 
Each finds a different niche, some will fail, some will 
take off. Common with acquisitions of growing 
companies, or the “house of brands” business model, 
or the “holding company” investment vehicle. 

¢ Multiple simultaneous solutions. To create iOS, 
Apple had the iPod team try to expand their OS to be 
more capable, while simultaneously having the 
Macintosh team try to reduce their OS to fit on a 
mobile device. One result was in-fighting, 
demonstrating that “cost” can be personal. But the 
result was that they selected the correct way forward 
for the iPhone. 

e Redundant systems. City power, and a generator. 
Multiple servers in different geographic regions. 


Teams, not solo engineers. Lying dials in airplanes. 


Redundancy is a cost, but the result is more 
predictable operation. As the military saying goes, 
“Two is one, one is none.” 

¢ Disrupt thyself. Create new products that disrupt 
your existing products. The quintessential example is 
Amazon launching the Kindle; Bezos instructed the 
newly-annointed Kindle leader—who was the leader 
over their book-selling business—that “Effective 
tomorrow, your job is to kill your old business with a 
Kindle.” Whether the future is in physical books, 


electronic books, or both, Amazon prospers. 


3 Wrap 3rd-party APIs in a proxy layer. Use multiple public 
clouds. Process online transactions with more than one party. 
Keep your cash in more than one bank. 


Extreme novelty 

Zappos, Airbnb, Uber, SpaceX, Tesla, Bitcoin, OpenAl, all 
took enormous risk by creating a new category that 
everyone else thought was impossible, even after they 
heard the plan. On one hand, this is the definition of un- 
predictability. On the other hand, they avoided the un- 
predictability that comes from existing competition in ex- 
isting markets. It doesn’t matter what’s happening at 
Amazon if you can return shoes for free even after 364 
days; it doesn’t matter how the hotel industry is shifting 
when you're selling a different experience; it doesn’t mat- 
ter what Ford and Toyota are doing with electric vehicles 
if yow’re changing the fundamental technology without 
the constraints of existing supply lines, dealer distribu- 


tion rules, and ties to the oil and gas industry. 


If everything is unpredictable anyway, why not earn 
strategic advantage: No direct competition, exciting place 


for top-talent to join, immense upside. 


That upside does need to be immense for the risk to be 
rational. If it is, this a strategy for trading into a better 


set of uncertainties. 
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Form coalitions 


If you want to go fast, go alone. If 
you want to go far, go together” 
—African proverb 


The more massive the object, the more it resists being 


moved by external forces. 


Price-collusion* 


removes the uncertainty of market pric- 
ing. Peace treaties prevent some wars. Open source 
projects attract more talent and advocates than any one 
organization could afford. Industry standards reduce the 
risk that core technology or protocols change, allowing 
members to build systems on top of those standard for 
decades, not having to “hedge.” Good things happen 


when we all agree to use HTTP HTML, and SSL. 


4 This is unethical and illegal in many countries; don’t do it. It is 
illegal because it works, which makes it a good example. 


Expand the scope of prediction 
Predicting exactly what will happen is folly, but mapping 


possible futures can be illuminating: 


Figure 1: The Futures Cone 


ossible 


laugible 


Probable 


Preferable 


Today 
uo 


“Futures Cone” from Voros 2003, expanded by Sjef van Gaalen 
in 2016 


Plotting possibilities that you believe are improbable, 
helps you recognize that perhaps they’re not quite as im- 
probable, helps you think of solutions that mitigate plau- 
sible challenges”, and help you recognize if an “improba- 


ble” thing is in fact happening. 


> The “pre-mortem” is an increasingly popular workshop for ac- 
complishing this, in which you brainstorm answers to the fol- 
lowing question: “It’s 12 months from now, and the project is a 
disaster. What went wrong?” The idea isn’t to solve everything 
you can imagine, but rather to pick a few things to intentional- 
ly mitigate, and intentionally leave the door open for mitigat- 
ing other things if they come to pass. 


Of course your'e still ignorant not only of probabilities, 
but of eventualities that you never predicted. Gaalen ac- 
knowledges this by further expanding the framework: 


Premeditated 


Possible Plausible Probable Preferable 


‘Black Swan’ 


‘Unknown Unknowns’ 


While this doesn’t mean you suddenly can predict the fu- 
ture, it might mean you’re paying more attention to 
what’s actually happening, allowing you to react faster, 


and thus to manage the unpredictability better. 


Stay simple 

The more tasks have to be broken-down, the more de- 
pendency arrows are drawn, the more intricate the analy- 
sis, the less believable the project estimate is. Complexity 


breeds unpredictability. 


The inverse is that simplicity is predictable. While not 
strictly true, is it true that simple things can be more pre- 
dictable. A simple product, with a simple value proposi- 
tion, in a simple market, has fewer dependencies that re- 


quire prediction, and thus are more likely to succeed. 


Bet on what will not change 

Made famous by Jeff Bezos, it is difficult to predict how 
the future will change, but it can be easy to predict the 
ways the future will not change. In Amazon’s case, he 
cites “low prices” and “fast delivery’ as two of those 
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things; in ten years, people will still want that. Therefore, 
Amazon can (and does) invest billions of dollars to 


achieve those results. 


In the case of WhatsApp, consumers want to chat and 
don’t want to pay; that was true ten years ago, it’s still 
true today, and it’s a good bet that it will be true ten 
years from now. In the case of WP Engine, the company I 
founded 13 years ago, people wanted websites to be fast, 
stay fast even when they get a lot of traffic, and be se- 
cure; they still do, and I’m sure they still will in another 


13 years. 


And so the lessons are: 


e Have a strategy, even though the world is 
unpredictable. 

¢ Decide quickly — get customer reactions quickly —> 
learn quickly > make new decisions quickly. 

e Upgrade the strategy by following customer behavior. 

e Product “abuse” is a strong signal for updating the 
strategy. 


e The strategy must include building a moat or two. 


e The strategy must create optionality in how to 
succeed (or avoid failure) so that single failures 
aren't fatal. 
building a product for a market for which you are 
already well-suited. 

e Either keep it simple, form coalitions, or do 
something completely novel. 

¢ Bet on things that won’t change, rather than 
predicting how things will change. 

¢ Do your homework, but don’t stall in analysis 
paralysis. Map future possibilities, to mitigate 
possible things and to better notice when your 


assumptions turn out to be incorrect. 


That’s how you win in an unpredictable world. 


Many thanks to Adam Brock, Amy Hoy, Daniel Zarick, Darla Cohen, 
Derrick Wolbert, John James Jacoby, KimSia Sim, Nitin Punjabi, Paul 
Huggins, Prasanna Krishnamoorthy, Rhys Jeffery, Seth Chasin, and Tony 
Meijer for contributing their insights to early drafts. 


Printed from: A Smart Bear 
https://longform.asmartbear.com/predict-the-future/ 
© 2007-2023 Jason Cohen W @asmartbear 
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Excuse me, is there a problem? 


by Jason Cohen on April 16, 2023 


Many startups fail despite identifying a real problem and 
building a product that solves that problem. This explains 
why, so you can avoid their fate. 


sy AndyWatson@hachyderm.io 


@andrewwatson 


Replying to @asmartbear and @mattwensing 


Dude the PTSD hit me so hard while reading this. I've 
heard all those excuses, made all those mistakes! 


oy AndyWatson@hachyde... @andrewwa... - 3h 


Replying to @asmartbear and @mattwensing 

| think the rubric is great because it gives you a 
framework for decision making. It reminds me of 
the old startup canvas. If i had this in 2011, 2013 
and 2016 it would have saved me a lot of blood, 
sweat and tears! 


How a lot of companies fail: 


— 


. Founder gets a flash of insight: The world has a 
Problem. 

2. Founder talks to three potential customers who are 
experiencing The Problem, or who are expert in the 
domain of The Problem. They agree The Problem 
exists. (And they’re correct!) 

3. Founder builds a product that solves The Problem. 
(And it really does!) 

4. Founder fails to make enough sales, and the company 
shuts down in 6-24 months, when the founder runs 
out of patience or money. 

5. Founder laments into the void (i.e. posts on Twitter): 

Why were sales so hard when the product clearly 


solved a real problem? 


These companies fail because solving a problem is—per- 
haps surprisingly—not nearly enough to build a success- 


ful company. 


The following model explains an extremely common rea- 


son why this happens, and what you can do about it. 


In the discussion, you'll figure out where your challenges 
are, and whether you can design a strategy to side-step 


the issue, or whether your business simply isn’t viable. 


Greatness needs luck, but 
it’s never by accident.” 


—Unknown 


The path from “The Problem” to “Viable 
Business Model” 

The main challenge facing a new startup is that so many 
different things have to go right for it to succeed. A sub- 
set of those things is the path “Problem” to “Viable 


Business Model.” 
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Plausible: Do 10M people or 100k 
companies have the problem? 


These numbers sound larger than necessary, but here’s 
why it is necessary even for an indie startup, using Fermi 


Estimation: 


1% conversion: Impression — Visitor 

An AdWords campaign with multiple keywords, ads, 
and bids would be very successful at a 2% click- 
through rate. Display ads are more like 0.3%-0.5%. 
(source: HubSpot) 


Top SEO position can be 3%-5%, but that’s almost im- 
possible to achieve for even a mildly competitive key- 
word. (source: HubSpot) 


1% conversion: Visitor — Paid 

A typical, successful product website converts 1% of 
its traffic to paying customers. I don’t have firm data, 
though I did an informal poll on this question years 
ago. Some data show 2-5% conversion rate even for 
just a sign-up form or free trial, of which a fraction 


will become paying customers. 


Therefore: 10,000 Impressions — 1 paying 
customer 

10,000 x (1% click-through) x (1% convert-to-pay) 
=1. 


10,000,000 Impressions — 1,000 paying customers 
Not every impression will be a unique person, but you 
still need closer to 10M potential eyeballs than to 1M, 
because while some people will see your material 
more than once, most of the market will never see 


your material. 


| needed for a 


1,000 paying customers is the minimum 
sustainable, small company. It will take about two years~, 


10M marketing impressions, and luck. 
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lThis is a rough rule of thumb companies charging 
$30-$100/mo; if less, you’ll need even more customers to be- 
come sustainable; if more, you need fewer customers, but they 
are more difficult to find and convert than the numbers above 
suggest. Pricing determines your business model. 


2 WP Engine, the company I started 13 years ago, was a hyper- 


growth company and then a Unicorn, reaching $100M in ARR 
in a similar timeframe to other hyper-growth companies (as in 
the chart mid-way through this article), now with over 200,000 
customers, yet it took two years to get the first 1,000 cus- 
tomers. Competitors who started after us also took that long, 


4 years, 6 months, or 52 hours, because this is a rule of thumb, 
not a law of physics. 


If you’re selling directly to consumers, there needs to be 
10M who have the problem, otherwise it’s too small even 


for an indie company who wants to stay small forever. 


If you’re selling to businesses, the total number of poten- 
tial customers is an order of magnitude smaller, but they 
will pay orders-of-magnitude more to solve problems, 


and conversion rates are higher, thus 100k is sufficient. 


A lot of great ideas attack problems that just aren’t actual 
problems, at least not for more than a small handful of 


people, and therefore fail to yield a successful company. 


Can you be successful anyway? Yes, there’s an excep- 
tion to every rule. For example, a high-price-tag product 
in a small niche can be a fine company. Or, perhaps 
you're happy staying frugal, never hiring an employee, 
and making $100,000/year post-expenses post-tax, re- 
placing a salary but on your own terms*, That’s wonder- 
ful. You can be the exception, but with conditions. 


3 This is, in fact, what 80% of small businesses do. It is a vibrant 
and valuable driver of fulfillment and the economy; ignore 
those who claim this is somehow less important than “swinging 
for the fences.” Rather, there are two kinds of companies: 
Those which endeavor to replace a salary (and then some), and 
those who are trying to become huge, and they are simply dif- 
ferent paths. 


Self-Aware: Do they know & care they 
have the problem? 


It seems like the answer should be “obviously yes,” but 


often the answer is “shockingly no.” 


If a person does not already believe they have a problem, 
they will not be surfing the Internet looking for a solu- 
tion, and even if they happen upon your website some- 
how, you cannot get them to spend money to solve a 
problem they don’t think they have. 


—_—_—_—_—_— 


SS 


\43 | DAYS 
WITHOUT 
ADDRESSING 
THE REAL 
PROBLEM | 


Ea 


Sam Altman is the co-founder and CEO of OpenAl, and 
before that ran the Y-Combinator accelerator, and is 
therefore one of the world’s most experienced experts on 
startups. He previously co-founded Loopt—a _ location- 
based, mobile social-network app. Oh look, those are all 
the keywords of 2005, when it was founded. It raised 
$30M and failed. When asked what happened, he said: 


The market wasn’t there. You can’t force a market. You 
can have an idea, and as a startup part of your job is to 
be ahead of it, and sometimes youre right about that, 


and sometimes you’re not. 


Sometimes you make Loopt, and sometimes you make 
OpenAI. You just keep trying. 
—Sam Altman, interviewed by Kara Swisher 


I’ve given the example of website security, which I know“ 


is a real problem plaguing millions of websites whose 
owners think “those mean hackers won’t attack lil’ ol’ me; 


” 


I’m nobody!” False. Hackers indeed don’t care about lil’ 
ol’ you, but they do want to gain control of your lil’ ol’ 


server, so they can do their nefarious things, like spam- 
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ming, advertisement-click-fraud, remote-controlling your 
visitors’ browsers, or just bouncing off to yet another 
server as a way of covering their tracks. Everyone has the 
problem, but millions of people don’t think they have the 
problem, so theyre not searching for website security 


software, and certainly not buying it. 


4 4 million websites run on WP Engine’s platform, serving tens of 
billions of requests daily, as 9% of the global online population 
visits a WP Engine property every day. We block hundreds of 
millions of nefarious requests daily, and internally run SOC 
Type II and ISO/IEC 27001:2013 certified security processes. 
So we know a lot about what hackers do to websites large and 
small. 


“Security” is a case of ignorance, but the other version of 
this challenge is when the customer knows they have the 
problem, but genuinely does not care. This could be be- 
cause this problem is the ninth-most-important priority 
on their list, and they can only give attention to their top 
three... and this item will never bubble up to the top 
three. An example I see a lot at WP Engine is accessibili- 
ty? Given lip-service by many marketing departments 
and product managers, it rarely makes the priority list for 
the public website or the product. Some companies 
choose to care, or are mandated by governments or con- 
tracts. This author chooses to care® But the fact is, 
though everyone agrees they have the “problem” of a 
non-accessible website or product, most don’t have the 


will to act. 


5 “Accessibility” means working well for people with various chal- 
lenges; in the visual sphere, consider cases like red/green col- 
or-blindness, needing high-contrast colors, needing larger text, 
or complete blindness, needing “screen reader” browsers to 
navigate menus, forms, and content. 


6 This site uses semantic tags for content and navigation, the en- 


tire stylesheet of both fonts and layout supports arbitrary 
changes in font size, automatically respecting browser-specific 
settings, supplies keyboard shortcuts for menu actions, has alt- 
text for all images, uses aria-title and related attributes, 
uses high-contrast foreground/background colors, also sup- 
ports high-contrast mode, and works well in screen-readers (in- 
cluding those built for everyone, like Safari Reader and 
Readwise Reader and Pocket). That said, let me know if there’s 
more I could be doing! 


Sometimes “willing and able” is a matter of market-tim- 
ing. A famous example is Instacart: Successful after 80% 


of Americans carried a smart phone, unlike WebVan 


which was exactly the same idea, solving the same prob- 
lem in much the same way, but the market wasn’t ready 
for it. 


A lot of great ideas, attacking real problems, fail to be- 
come successful companies, because the target market 
doesn’t know they’re even in the market. Because they’re 


not. 


Can you be successful anyway? Yes, there’s an excep- 
tion to every rule. Some founders are not only the first- 
and-best sales-person, but also natural evangelists. More, 
they’re on a mission to educate the world about their pas- 
sion. They don’t see a lack of interest as a barrier, but as 
an opportunity to change minds. That is a difficult, ex- 
pensive, and slow path’, but it is a path, and one that 
could result in zealous, loyal customers and a fulfilling 
existence. But you really have to want that path, doing 
that work with those consequences, if you’re going to en- 
ter a market that you also have to create. 


7 “Difficult” because changing someone’s mind about anything is 
almost impossible, especially when they're not seeking to have 
their mind changed. “Expensive” because of the marketing and 
attention you have to bring to the cause, on top of the usual 
work of making a sale, with certainly-worse-than-average con- 
version rates. “Slow” because you’re having to create demand 
and then fulfill it, rather than meet demand that already exists. 


Lucrative: Do they have substantial 
budget to solve this problem? 


e “You have a great product! I’d have to get [someone 
else’s] approval for this though, and they don’t 
understand all this.” 

e “This is nice, I would use it, but they’re carefully 
watching all expenditures and the truth is I can 
manage without it.” 

e “I'd love to, but our budget is closed for the year and I 
can’t start a new project.” 

e “This is pretty cool, but our internal team who 
manages [the problem] says they don’t need help. They 

might be just trying to save their jobs, but it is what it 


” 


1S. 
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We've all heard these objections. Some are normal; you 
can’t win every sale. But sometimes the target customer 
agrees they have the problem, yet doesn’t have the mon- 


ey to solve it. 


At Capital Factory® there’s a constant stream of kids com- 
ing out of college with a startup that “sells to col- 
lege students.” It’s easy to find “problems”—restaurants 
and bars want to advertise to students, students don’t 
want to spend much on food, students need books and 
supplies, and so on. The founders explain they “had the 


problem themselves, so they really understand it.” 


8 Capital Factory is by far Austin’s largest and most prolific “Cen- 
ter of gravity for entrepreneurs in Texas,” now a tiered system 
from co-working to mentoring to multi-million-dollar invest- 
ments with hundreds of companies in orbit. The University of 
Texas, also in Austin, has a high-quality Computer Science de- 
partment that is also one of the largest in the country. 


That’s probably true. The deal-breaker is that college kids 
have no money, and don’t spend what little money they 
have on SaaS products. And businesses that cater to col- 
lege kids have to charge low prices (because college kids 
have no money), and therefore don’t have budgets for 
oddball new ideas. In 15 years, none of these startups 
worked, even though arguably most of the “problems” 
existed. 


You might think a large company will definitely allocate 
budget for a known problem, but here again the answer 
is often in the negative. Budgets are applied to the top 
few most-important problems of the year; if this is a 
problem, but not a top one, it won’t get attention. Large 
companies have to allocate more than money—they have 
compliance teams who have to approve, they compare 
multiple vendors, they run pilot programs, and all of this 


won't be set in motion unless it’s a top problem. 


Large companies have internal teams that are already 
tasked with the problem, which might means there’s no 
additional budget for outside solutions. Those teams of- 
ten fight against outside tools, because they makes their 


jobs obsolete, or at least converts them into vendor-man- 


agers instead of innovators. You want to target compa- 
nies who outsource this particular problem to outside 
vendors. 


Once you get over the hurdle of there being a budget at 
all, is the budget large enough? I’m always shocked how 
little people will pay for productivity applications like to- 
do lists and note-taking. These are applications you might 
use dozens of times per day, as much as email. The slight- 
est increase in efficiency or even simple delight will have 
a massive impact on the customer’s life, every day. And 
yet people complain that the Pro version of Bear App’ is a 
whopping $15/year, or that Remember the Milk is 
$40/year, or how they've been paying €29/year for 
Evernote for eleven years, but a change to €43 is so dev- 
astating that they will completely change to another ap- 
plication that has 1/10th the functionality and no tech 
support: 


9 My note-taking application of choice, and not just because it’s 
called “Bear! “ 


2 geldnerd - 19 days ago 


I've been on Plus since 2012 for € 29 per 
year, never increased since. Saw in my 
account details that it would be increased 
to € 43,50 per year upon next renewal. So 
I moved to Joplin, emptied my Evernote, 
and cancelled my subscription and 
account. Check your account details... 


In general, consumers don’t like paying for stuff, hence 
the multi-trillion-dollar success of having people “pay” 
with attention (advertisement) and data (privacy). This is 
why I think?° self-funded companies in particular should 
target businesses as customers; unlike consumers, they 
will spend money to solve problems and to make more 


money. 


10 This is clearly a personal bias. I can’t wrap my head around the 
mentality described above, and that’s why I don’t build in and 
generally don’t invest in B2C—I’m fully aware that I don’t un- 
derstand the customer! 
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A lot of great ideas, attacking real problems that cus- 
tomers acknowledge and seek solutions for, are in areas 
where budgets don’t exist, or not often, or are so small 
that it requires an enormous number of customers to 
make money (often also in a crowded competitive space), 


and therefore the company fails. 


Can you be successful anyway? Yes, there’s an excep- 
tion to every rule. If there are a huge number of poten- 
tial customers, and if your cost-basis is extremely low! 
you can create a strategy targeting a large market at a 
low price with a simple product!” This works even better 
if existing products are poor (so you can stand out and 
make a splash) and expensive (so your low price is itself 
a differentiator). It’s still risky and difficult, but you could 
accept that and make decisions consistent with that chal- 
lenge. But you have to really want that path, and in- 
crease the “10M” number, since you'll need a lot more 


customers to make ends meet. 


lleg. no direct customer service, no substantial infrastructure 


costs, a route to market that costs almost nothing to acquire a 
customer, the ability to build and maintain the application with 
very little design or engineering or product outside of the 
founders, switching costs are low so you don’t have to do a lot 
of work or spend a lot of money to get a customer off another 
product and on-boarded onto your product 


12 when a product will be widely used with little-to-no customer 
service, it must be simple or it won’t scale. 


Liquid: Are they willing and able to buy 
right now? 


e “Oh yeah, we spend $100,000/year on this. But our 
contract isn’t up for renewal for another 15 months.” 

e “We just implemented a version of this in Workday. I’d 
rather use your product actually, but it’s just part of 
Workday and the HR team likes that everything is 
integrated.” 

e “This is way better than our current system, but we’ve 
invested a lot integrating with seven other systems, plus 
a few custom things some engineering teams did, so we 
can’t really consider switching away at this point.” 

e “That looks like great software for weddings. I'll let you 


know when I get married again! Haha!” 


When the customer is already paying to solve the prob- 
lem, or actively comparing options to solve the problem, 
you still run into the barrier of whether they have the or- 


ganizational will to buy from you. 


“We found the problem — all of your internal 
organs are in this little bag.” 


This can be for legal reasons, like being locked into a 
long-term contract or government fiat. This can be for 
convenience, as in the Workday example which at WP 
Engine caused us to cancel several other SaaS products 
because “now it’s all in one system, which we’re paying 
for anyway, so this is simpler and safer to manage.” This 
can be because of other forms of lock-in, like difficulty in 
extracting and moving existing data, having to retrain 
thousands of employees, or having to re-implement a va- 
riety of cross-systems integrations that people rely on for 
their workflows, data, reports, and governance. 


Notice that all these forces have nothing to do with 
your product or its price. They are so powerful, they 
overwhelm a product that is solving the problem the best, 
at the best price. That’s why they cannot be ignored, and 
why founders are surprised when their genuinely-great 
product in the definitely-extant market where customer 
are definitely paying for solutions, is still too difficult to 


sell. 


On the bright side, this is a prompt for your strategy. How 
will your strategy create these sorts of “lock-ins” that will 


prevent your competitors from kicking you out of your 
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customers? 


A final way that customers might not be able to buy right 
now, is when the product is needed at a specific moment 
in time, but not before or after that moment. Websites for 
events and occasions are an example, as are tools that 
solve temporary problems like sophisticated code profil- 
ers or load-testing tools. Because Smart Bear !* was in the 
developer tools market, I know a number of founders of 
products in the latter two spaces; all of them struggled to 
maintain even small companies, exactly because people 
didn’t proactively need the product (i.e. “Don’t have the 
problem”), but then suddenly did (ie. “Willing to buy, 
but only right now”), and then didn’t any more (i.e. “No 
longer willing to buy”). 


13 My company before WP Engine and the namesake of this 
website. 


Can you be successful anyway? Yes, there’s an excep- 
tion to every rule. If there’s a legal contract, you can of- 
fer to pay the penalty for them breaking their contract 
(so long as that isn’t so expensive that you can never be 
profitable). If it’s hard to migrate, you can offer to do the 
migration as a service, perhaps even for free (this is com- 
mon in WP Engine’s market). If you’re fighting the all-in- 
one enterprise systems like Workday, you can focus in- 
stead on a target market that cannot afford Workday and 
sees Workday as overly complex and cold, so that 
Workday isn’t even a competitor. If timing is important, 
you might offer a way to buy that is cheap or even free 
while not using it, so you’re “right there as soon as you 
need us again.” But you have to really have answers for 


these challenges, and be ready to walk that path. 


Eager: Do they want to buy from you, 
specifically? 


¢ “It seems like it would work for us, but it looks like 
you've only been in business for a year?” 

e “Tt definitely worked fine during the trial, but we’re 
expecting to grow 100x and we’re not confident that 
you'll be able to handle it.” 


e “For the features we need, [competitor] looks the same 
to us, and they’re cheaper.” 

e “I like how you do X, but [competitor] does Y and Z, 
which we really like, so we’re going with them.” 

e “Our policy requires that all vendors are SOC 2- 
compliant and provide a security audit trail API, so you 


did not meet our basic requirements.” 


Even in a real market, with customers spending real mon- 
ey, in a purchasing process right now, you haven’t yet 
won. There are other options, ranging from direct com- 
petitors to indirect alternatives. They will buy, but will 


they buy from you? 


"It's not that I don't like you, Ted, you just don't 
fit my target demographic." 


The first hurdle is trust: Do they trust not only that the 
product works, but that your company will be around for 
many years to come, that you will maintain a high pace 
of development, that you won’t have security issues, that 
your customer service will truly help them when in- 
evitable problems arise, and that you can scale as the cus- 


tomer’s needs scale? 


The second hurdle is differentiation. This doesn’t just 


G 


mean “you have something unique.” You might have a 


feature that no one else has, but if only 10% of the mar- 
ket cares about that feature, that’s not enough. Worse, 
your competitor will have some feature you don’t have, 
and what if 30% of the market cares about that one? 


How to do this? See this companion article on leverage. 
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A special difficulty comes when the product over-serves a 
large segment of the market. This means that, for exam- 
ple, you have ten features, but most of the market really 
cares about three of them. You might have all sorts of dif- 
ferentiation in the latter seven, but that won’t sway most 
people. Picking something simpler or cheaper is the ratio- 
nal choice, even though your “feature comparison ma- 


trix” shows that you’re a much more complete solution. 


A sales team can combat both of these challenges. The 
job isn’t just to schedule calls, cajole potential customers 
into action, and press for the close; the job also to build 
trust in your organization and talk around competitive 
points, positioning so that the customer wants to buy 
from the mixed bag of plusses-and-minuses that every 


product contains. 


The best way to overcome these challenges are the “Love” 


and “Utility” types of willingness-to-pay. 


Can you be successful anyway? Yes, there’s an excep- 
tion to every rule. Trust can be side-stepped by building 
a type of product that doesn’t require much trust‘4 Or 
trust can be built by mitigation, for example open-source 
means the customer can shift to another vendor or take 
over the code base in a _ worst-case scenario. 
Differentiation is harder. Sometimes you’re competing on 
price, which isn’t ideal for the bottom-line or for the qual- 
ity of customer, but can work very well. In a large mar- 
ket, differentiation can come from specializing in a niche. 
In a small market, you might not have many viable com- 
petitors, lessening the importance of differentiation. A 
company mission that is “bigger than all of us” can also 
be a distinguishing reason to buy?>, although typically 
more for consumer products where that’s an allowable 
purchase-criterium. Still, it difficult to survive when 
you're no different from more mature, feature-rich, sta- 


ble, innovative alternatives. 


14 “Security” isn’t a concern if the data in your app isn’t private, 
such as a social media management tool. “Uptime” isn’t a con- 
cern if the product is run locally or the service isn’t time-criti- 
cal. “Company maturity” isn’t a concern if it’s a tool for individ- 
ual use; in fact it can be an advantage for a new startup to sell 
to freelancers or other people who want to support other 
startups. 


15 People buy windbreakers from Patagonia that are undifferenti- 
ated from other outdoor apparel vendors, because Patagonia is 
well-known for spending hundreds of millions of dollars on 
conservation and sustainability, and for treating their employ- 
ees well. 


Enduring: Will they still be paying (or 
paying-it-forward) a year from now? 


I cannot count the number of indie developers who grow 
to $15k MRR and start slowing down because their can- 
cellation rate is 7%. Many don’t think this is a problem; 
many of the rest believe that 5% would be a success. It’s 
not. 


Financials and SaaS metrics aren’t even the primary rea- 
son why this is fatal. The problem is that the customers 
don’t want the product. 5%/mo cancellation means only 
half the customers will still be customers a year from 
now, which means you're not building a sustainable 
business. 


The reasons can vary. Perhaps they needed the product 
temporarily; they might have loved it, but “the problem” 
disappears. More often, the product didn’t work well 
enough—insufficient features, too many bugs, didn’t inte- 
grate with some other system, too expensive for the end 


result, turns out the problem is not important enough. 


The financial reality is illuminating too. The challenge is 
that top-line growth is linear for many companies, and 
panies. But cancellation is exponential—that’s why it’s 
measured as a percent of the current customer base. 
Exponentials grow faster than lines or quadratics, there- 
fore cancellation “catches up” faster than you can add 
new customers. Growth fades, and finally ceases, as one 
customer cancels for every new customer who signs up. 


With $15k MRR, adding $2k/mo of new customers—a 
healthy 15% per month growth rate—a 7% cancellation 
rate means already half of that growth is negated by cus- 
tomers leaving. The company barely got started and al- 
ready its growth is being decimated. At that rate, only 
one year later, having grown to about $27k MRR, the 
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company has stopped growing completely!® despite 
spending time and money on marketing and sales. Don’t 
forget—those new customers cost money to attract, sell, 
and on-board with tech support, but all the value of that 
expense is negated by an equal number of customers 


walking out the door. 


16 Growth stops because the $2k of new customers arriving in a 
month are negated by $27k x 7% =~ $2k customers cancelling 
in that same month. 


Revenue 


| MRR @Churned Revenue ® Growth Rate 


Charting this scenario using the SaaS Plateau Forecaster from 
Summit. 


You certainly aren’t required to have a goal of “grow 
forever’’,” but capping growth because customers don’t 
really value your product, is not a healthy business no 
matter what your end goal is. 


17 But especially if you do have the goal of becoming a scale-up 
Unicorn, it is impossible to do that without low churn. As just 
one example, Gainsight CEO Nick Mehta recently pointed out 
why “This stall will happen to all companies eventually,” ex- 
plaining that launching a second product was their solution. 


This example was for a recurring-revenue business, but 
the same principle is true for one-time revenue business- 
es. One-time revenue businesses still require repeat 


revenue, in two ways: 


1. Customers buying again. 
2. Happy customers telling other people to become 
customers. 


Both of these require that customers are happy with the 
product. This reinforces the point that the most impor- 
tant problem is that the customer isn’t satisfied, regard- 
less of business model. 


Can you be successful anyway? Yes, there’s an excep- 
tion to every rule. Shopify is a fantastically successful 
business with a 8% cancellation rate—only 34% of new 
customers stick around for one year. They’re successful 
anyway because (a) the ones that do stay tend to grow 
forever and super-linearly, (b) the market is enormous 
and still growing fast, so they won’t run out of new cus- 
tomers anytime soon, and (c) customers cancel because 
their own business didn’t succeed, not because Shopify’s 
product is problematic. If the problem really is the prod- 
uct, I don’t believe there is an exception. The product has 
to be work well for some segment of the market. The rest 
can be mitigated if other factors of the market, strategy, 
and business model can overcome this massive weakness 


with even-more-massive advantages. 


Evaluating startup viability 


This path of “problem” to “business model” is not the 
only factor that determines success. There are still ques- 
tions like whether you can reach customers, get their at- 
tention in the noisy Internet and competitive space, can 
you do that cost-effectively, are company costs too high, 
do you have the skills, can you hire for skills, do you have 


enough time and money to do it, and so on. 


Still, we can evaluate the viability of this path with the 
following model. We’ll use Fermi Estimation to avoid the 
analysis-paralysis of deep research and arguing over 
details. 


Some people have already started using this to think 
through their businesses, like Sam Bhagwat, co-founder 
of Gatsby, thinking through his next startup. There’s also 


some online calculators if you like that sort of thing. 


Answer with regard to a specific target market, which 
means a specific type of buyer solving a specific problem 
with a product that has made specific trade-offs, at a spe- 


cific price. 
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Criteria 

Plausible 

Number of potential cus- 
tomers (consumers or busi- 


nesses) who actually have the 


problem 
Self-Aware 
Willing to solve the problem 


Lucrative 
Annual allocated budget 


Liquid 
Frequency of purchase 
decision 


Eager (identity) 
Attitude towards your 
company 


Eager (comparative) 
Competitive differentiation 


Enduring 
Will they still be here a year 
from now? 


Normalize 

Normalize the score so that 
1 is the threshold for an in- 
die startup, 2 or more for a 


scale-up. - 
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Score 

Power-of-ten only 

1k, 10k, 100k, 1M, 10M, 
100M, 1B 


0.01: Few agree or care 

0.1 : Thought-leaders care 
and evangelize 

0.5 : It’s an industry stan- 
dard-practice 

1.0 : Almost impossible to 
find someone who doesn’t care 
Power-of-ten only, of net-rev- 
enue 

$1, $10, $100, $1k, $10k, 
$100k, $1M 

0.01: Every few years 

0.1 :An annual decision 

1.0 : Always in the market, 
easy to switch 

0 : They cannot buy from 
you 

0.1 : Structural challenges 
0.5 : Indifferent; no red flags 
1.0 : Mission-level emotional 
desire to select you 

0.1 : No material differentia- 
tion 

0.5 : Some features are so 
good, some people will buy 
just for that 

1.0 : One-of-a-kind solution 
that has no viable alternative 
0.01: One-off purchase with- 
out loyalty 

0.1 : One-off purchase, but 
happy customers will buy 
again and tell their friends 

0.5 : Recurring-revenue from 
a recurring-problem 

1.0 : Strong lock-in from fiat, 
integrations, or being the sys- 
tem-of-record for a business- 
critical system 

Divide by 625,000 


Now you multiply. Why multiply? Because this is a series 
of “ands”—there needs to be customers with the problem 
and they have budget and they are buying today and so 


on. The effects compound. 


18 “Net-revenue” means your revenue after subtracting pass- 
through costs. For example, an eCommerce platform might 
process a $100 purchase on behalf of its customer, keeping $10 
for itself as payment; that is net-revenue of $10. Pass-through 
costs do not include cost-of-goods-sold, i.e. you should not sub- 
tract out the marketing and sales costs to acquire the customer, 
nor customer support, nor infrastructure costs for SaaS prod- 
ucts. Those are important for profitability, but in this exercise 
we are focused only on top-line revenue, not on the efficiency 
of your operations. 


19 Justification: Using the figures earlier in the article, you could 
be successful with 10M consumers at $10/mo, or 100k busi- 
nesses at $1000/mo (e.g. dentist practice-management), so 
consider the threshold of those numbers combined to be 100M. 
Taking the middle value of all other questions—neither a deal- 
breaker nor a_ strong advantage—you end up _ with 
100M x 0.5 x 0.1 x 0.5 x 0.5 x 0.5 = 625k. Arguably 
you should have some strong advantages, but also some of 
these values will be on the low side, so this is a reasonable 


Fermi-style acceptance threshold. 


This is dangerously close to a silly quiz or rubric, so 
we have to be careful to use the final score as guidance 
rather than precise analysis. Still, different choices of tar- 
get market, target customer, and product trade-offs can 
result in dramatically different results. 


As usual, having to think through the answers and 
trade-offs is most of the value of the exercise, even 
more than the final score. 


A few examples 
Still, let’s try it, using WP Engine”” as the example. Note 
how our research is simplistic, but because we only have 


to be accurate to a power of ten, the answers are easy 


anyway7!: 


20 WP Engine is the largest platform for WordPress-based websites 
among the top ten million websites in the world. Therefore, 
we'll take the market as “businesses using WordPress.“ 


21 For example, our figures for the first row are just page-one 
Google search results, but even if the figures are off by 50%, 
certainly the true answer must be far larger than 10M and far 
smaller than 1000M, hence 100M is the easy choice. 
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Criteria Score Justification 

Plausible 100M There are 334M businesses in the 

Number of busi- world, 71% of which have a website. 

nesses who 43% of websites are WordPress. 

have the 

problem 

Self-Aware 0.1 While everyone using WordPress de- 

Willing to solve finitionally hosts it somewhere, and 

the problem industry practitioners often use a 
specialist vendor, most target cus- 
tomers don’t care enough to do 
more than the bare minimum. 

Lucrative $100 Difficult to say over such a large 

Annual allocat- market; one could argue that $10 is 

ed budget more accurate because most busi- 
nesses are small and most don’t buy 
expensive American things. 

Liquid 0.01 People rarely change their website 

Frequency of platform. 

purchase 

decision 

Eager (identity) 0.5 Could argue that itis 1.0 today be- 

Attitude to- cause of our leadership position, but 

wards your ten years ago we were one option 

company among several, all of which were vi- 
able for many customers. 

Eager (compar- 0.5 We have many capabilities and fea- 

ative) tures that are either unique or we 

Competitive are the best, but competitors have 

differentiation other advantages, whether in fea- 
tures or price or geography. 

Enduring 1.0 Today we easily justify this with our 

Will they still be world-class retention metrics across 

here a year 14 years of customer data, however 

from now? even early on we saw high retention, 


and comparables also have high 
retention. 


Score = 2,500,000 / 625,000 = 4, so it qualifies as a 


scale-up, and indeed that’s what happened. 


Criteria Score Justification 

Plausible 10m There are tens of millions of newslet- 

Number of ters using competing products. 

people who 

have the 

problem 

Self-Aware 1.0 This is well-known as a best-practice, 

Willing to with multiple at-scale companies 

solve the serving a mature market. 

problem 

Lucrative $100 Customers with complex workflows 

Annual allocat- and many thousands of subscribers 

ed budget will pay more, but most customers 
aren’t in that category, and might pay 
$9/mo.22 

Liquid 0.01 Customers buy newsletter software 

Frequency of and then want to just use it, not be 

purchase switching 

decision 

Eager (identi- 0.5 The founder was well-known among 

ty) other indie founders, however in the 

Attitude to- broader market there’s no reason not 

wards your to trust them, but no particular mis- 

company sion-driven reason to pick them 
either. 

Eager (com- 0.5 Most features are similar, but there 

parative) are interesting things at the margins 

Competitive for routing readers through flows and 

differentiation monetizing content, whereas more 
generic newsletter products don’t 
have as many tools for direct 
monetization. 

Enduring 0.5 For the customers who avail them- 

Will they still selves of complex automations, this 


be here a year 
from now? 


would be a 1.0; most people will 
probably use it as a normal news- 
letter, with a simple follow-up flow 
that competitors also provide. 


Let’s try it again with an indie startup: ConvertKit, a 
email marketing product competing with giants like 
Constant Contact and newcomers like Substack, designed 
to help you grow and then monetize your subscribers. 
They target creators who want to monetize their newslet- 
ters, not “anyone and everyone,” which reduces the tar- 


get market but increases willingness-to-pay. 


22 Alternately, you could focus on the customers who pay more; 
you might then reduce the “number of people who have the 
problem” to 1m, and the annual budget to $1000. While re- 
sults in same score, it’s a different product, serving a different 
market, so this is an important decision. 


Score = 1,250,000 / 625,000 = 2. This is a good busi- 
ness model, possibly even a scale-up, and indeed 


ConvertKit grew quickly as a bootstrapped company, and 


11 of 13 


Excuse me, is there a problem? 


while not a Unicorn, is an order of magnitude larger than 


most small businesses ever become. 


Finally, let’s take the case of selling security software to 
consumers. This is a tough market; there are success sto- 
ries (e.g. Norton, 1Password, Cloudflare), but it’s hard to 
find small indie companies who are successful in this 
area (whereas it’s easy to find successful indie WordPress 
hosting companies): 


Criteria Score Justification 


Plausible 1B There are ~5B people online today, 


Number of peo- who might have online security 


ple who have concerns 
the problem 
Self-Aware 0.01 Have you bought special security 


Willing to solve software for an online project? 


the problem Almost no consumer does. 


Lucrative $10 Consumers don’t pay much for 


Annual allocat- things, and globally consumers 


ed budget spend far less on average than 
Americans on equivalent online 
goods; arguably this should even be 
$1. 

Liquid 0.01 Consumers are rarely in the market. 

Frequency of 

purchase 

decision 


Eager (identity) 0.5 Unclear without specifics, we'll be 


Attitude to- generous and assume you can earn 


wards your trust despite being an unknown 
company brand. 

Eager (compar- 0.1  Consumer-grade security products 
ative) are undifferentiated. 

Competitive 

differentiation 

Enduring 0.5 Consumers cancel at higher rates 
Will they still be than businesses, but at a low price 
here a year this could be something that is easy 
from now? to maintain; industry data would be 


helpful. 


Score = 25,000 / 625,000 = 0.04. This is not a good 


business model. 


But there’s hope... 


What to do with a negative answer 


“This is why no one likes math; it’s a branding issue. 
Everything’s a problem.” 


We don’t have to give up just yet. 


What if the security company targets high-net-worth indi- 
viduals instead of “everyone?” The “number of people” 
would fall to 1,000,000, but willingness to solve the 
problem might rise to 1.0, and budget certainly rises to 
$100 if not $1000. The orders of magnitude can change 


dramatically, which might reveal a workable model. 


What if the security company targets mid-sized business- 
es? The number of organizations is smaller than the num- 
ber of consumers (but is still large), willingness to solve 
is very high (they have security policies and fear of the 
downside of a security incident), budgets are substantial, 
and so on. 


In general, targeting a niche often results in a better 
business model, because although it reduces the number 
of target customers, it can increase several other num- 
bers. This is very often the right answer whether your 
goal is to build a small, profitable, sustainable business 
(in which you stay in that niche forever) or a large multi- 
billion-dollar enterprise (in which the niche is your way 
to get started, and you expand the target market over 


time). 
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e The factors that cause customers to be willing to pay 


I'd rather be wrong than do more for a product. 
something wrong. is e Trading off many customers at low price-points 
—Larry Ellison versus few at high price-points. 


e Why selling to the mid-market can be the “worst of 


both worlds.” 
Or, finally, your business idea might simply not be vi- e How small companies can win against Enterprise 
able. That is a sad and tough reality to face, but it’s bet- incumbents. 
ter to figure that out early, so you can spend your time ¢ Selling to Carol: How targeting your one “perfect” 
finding a new idea. customer is the right way to market in general. 


I hope this framework helps you build a winning ; ; oe : 
Many thanks to Daniel Zarick, KimSia Sim, Matt Wensing, Sam Bhagwat, 


strategy! Tony Meijer, and Willis F Jackson III for contributing their insights to early 
drafts, and to Daniel Veihelmann for the online calculator. 


Further Reading 


Printed from: A Smart Bear 
e Five ways to build a $100M business, the classic by https://longform.asmartbear.com/problem/ 
Christoph Janz that uses five animals to show the © 2007-2024 Jason Cohen W @asmartbear 
different orders-of-magnitude of price versus size of 
company and quantity of those companies. This is 


equivalent to two of our rubric lines. 


Five Ways to Build a $100M SaaS Business 


1,000,000 


100,000 


GL 


# of 10,000 (cle) 
customers 


1,000 


T > 


T T 
$100 $1,000 $10,000 $100,000 $1,000,000 


ARPA 


(Average Revenue Per Account per year) 


¢ Pricing determines your business model: How orders- 
of-magnitude in pricing changes your product and 
target market. 

¢ How to use the Needs Stack to determine both 
features and benefits that make sense for your 


customers. 
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Willingness-to-pay: Creating permanent competitive advantage, for 


the right reasons 


by Jason Cohen on May 21, 2023 


This fresh take on “Willingness-to-Pay” analyzes three types 
of customer motivation, leading to superior strategies for 
growth that also better the world. 


“OK, let’s not get into profit shaming here.” 


Traditional economics: WTP and 
Consumer Surplus 


The best businesses deliver $4 of value, charge $2, and 
costs them $1 to do it. 


It’s an obvious formula for both profit and happy cus- 


tomers, but what does “$4 of value” even mean? 


An economist labels that formula as follows: 


PROFIT CONSUMER SURPLUS 


COST INCURRED 
PRICE CHARGED 


WILLINGNESS-TO-PAY (WTP) 


l l I I l 
$0 $1 $2 $3 $4 


Willingness-to-pay (WTP) is the maximum price the cus- 
tomer would have paid for the product, which the econ- 
omist claims is how much the customer values the prod- 
uct. “Value” could come from anything—utility, pleasure, 


status, even irrational confusion. The economist claims 


that any transaction is evidence that WTP > Price, and 
the difference between those numbers is “Consumer 


Surplus.” 


It looks trivial at a first glance, but I’ve come to believe 
that analyzing “WTP” is not only non-trivial, but also 
leads to very different strategies, business models, and 


outcomes. 


“Willingness” to pay 


I'm irked by this word “willingness.” 


In 2015, Martin Shkreli, then-CEO of Turing 
Pharmaceuticals, bought the rights to the drug Daraprim, 
which for 62 years had been used to treat a deadly para- 
sitic disease. He raised the price of a pill from $13 to 
$750, skyrocketing the typical cost of treatment from 
$1,000 to $63,000. 


Martin Shkreli testifying before congress on a hearing on drug 
prices, before calling lawmakers “imbeciles” 


“Profit” was his only justification for this abuse, in his 
y J 


own words: 
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“T think it will be huge.... So 5,000 paying bottles at 
the new price is $375,000,000—almost all of it is prof- 
it, and I think we will get 3 years of that or more. 
Should be a very handsome investment for all of us.” 
—Martin Shkreli, in communication with investors 


Patients have no choice: It’s pay or die. The economist 
would say, patients objectively have a high “willingness” 


to pay. But is this how we should define “willing?” 


And when patients cannot afford a $63,000 treatment, 
and therefore don’t purchase the drug, and die, should 
we say “well, I suppose they weren’t ‘willing’ to pay?” 
This phrase captures neither the intent nor the ability to 
pay, both of which are critical factors in questions of 


price, profit, and consumer surplus. 


While there are many! such examples, it’s more instruc- 
tive to point out mundane, non-life-threatening examples 


of why “willing” is not the right word. 


1 There were at least four egregious cases in 2015 alone. More 
recently, Moderna quadrupled the price of their COVID vac- 
cines, its CEO Stephane Bancel saying that the new price is 
“consistent with the value” of mRNA vaccines at 45 times the 
manufacturing cost, after the US government paid them bil- 
lions to cover the cost of developing the drug. Are we “willing” 
to pay even more? “Yes” in the sense that human life is valu- 
able, but “no” in the usual sense of the word “willing.“ 


It happens with commodities, which economists say are a 
“perfect market.” When crude oil prices go up, prices at 
the pump go up immediately, even though costs haven’t 
yet risen. When crude oil prices go down, prices at the 
pump go down slowly, even after costs have in fact fallen. 
The same thing is happening now with eggs. Is that be- 


cause we're all “willing” to over-pay for gas and eggs? 


It happens with bundling—often touted _as_ a wonderful 
strategy. I never liked paying for cable TV because it 
seemed expensive considering I still had to watch ads all 
the time. Most of the channels I paid for, I didn’t watch. 
Cable companies know that of course; they bundle chan- 
nels specifically because they know consumers are not 
“willing” to pay for all of them. Because the content-own- 


ers have a near-monopoly, consumers have no choice. 


Even with modern streaming services the problem per- 
sists, because whether it’s Hulu Live or YouTube TV, it’s 


still bundled, and still the same price. 


"I prefer to call it profit sharing." 


There’s also “willing” versus “able.” Perhaps many more 
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consumers would be “willing” to pay $1000 for a fully 
tricked-out smartphone, but most are not “able.” This is 
vital fact when determining strategy, business models, 
and company viability, but an economist would just say 
“few consumers are ‘willing’ to pay $1000 for a high-end 


phone.” 


But it’s not all bogus. There is a genuine concept of being 
“willing” to pay more, and thus genuine “Customer 
Surplus.” I am willing to pay more for Anker products 
(power strips and chargers) because they’re extremely 
high-quality; I don’t even notice if there’s a competing 
product that is 20% cheaper. I’m loyal even though there 
is neither lock-in nor recurring revenue. People pay more 
for TOMS? and Patagonia® products because of the au- 
thentic missions of the companies. People routinely pay 
more for coffee that has a fair and sustainable supply- 
chain, because they’re truly willing to pay more to have a 
positive impact on the world, not just to consume the 


product. 
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2 Blake Mycoskie was vacationing in Argentina, when a knowl- 
edgable American opened his eyes to the outsized impact that 
a lack of shoes has on poor children. Unprotected feet are sus- 
ceptible to punctures and infection, and prohibit walking long 
distances, which in turn means one cannot go to school. He 
founded TOMS shoes, selling an Argentinian-style shoe, with 
the logo of the Argentinian flag, with a marketing strategy he 
dubbed One for One: Every time you buy a pair of shoes, 
TOMS would give a pair to a needy child. After TOMS’s finan- 
cial success, Sketchers copied the strategy exactly, even down 
to the style of the shoe, the name (“BOBS”), and the altruism. 
Consumers were so outraged by this inauthentic strategy, 
Sketchers was forced to canceled the product line after just 24 
hours (although they revived the brand later with a different 
mission). That strategy was individual to TOMS; it was irrele- 
vant that the strategy was publicly visible and copy-able. TOMS 
has weaknesses—people complain about poor customer service 
and shoes quickly developing holes—but they win anyway on 
the strength of the individualized story. 


3 Besides their publicly-lauded sustainable practices and an out- 
door-worshipping culture, they even have a formal company 
policy to bail employees out of jail if arrested while protesting 
peacefully. 


Indeed, genuine “willingness” creates the best, most 
durable, most profitable businesses. Consumers not only 
pay more, they’re happy to pay more, creating profit mar- 
gin. They become evangelists, driving efficient growth. 
The company is resilient to competition, because con- 
sumers are buying for reasons other than “features” and 
“price.” The world becomes a better place, transcending a 


zero-sum game of winners and losers. 


Analyzing the differences between these kinds of WTP 
yields insights that all products and companies can lever- 


age to build the best strategies. 


Three kinds WTP 


I divide WTP into three categories, each having different 


drivers, and much different strategic value: 
Love 


e Mission: the joy of supporting a change that’s bigger 
than all of us, or a community or movement you 
want to see flourish 

e Reciprocity: when the company gives before taking, 
or gives more than it takes, or provides exceptional 
customer service 


e Exceptional design: a joy to use, a product that seems 
to genuinely care about your experience 

e Exceptional quality: the pleasure and relief generated 
by reliability 

¢ Personal identification: leveraging the company’s 
brand as visible component of your own personal 
brand 

¢ Culture: supporting an organization that treats 
employees and vendors well* 

¢ Social or environmental impact: supporting 
sustainable, fair practices 

e Community: a welcoming space where members 
learn and teach, support each other, create personal 
connections, grow their career or business, be part of 
a tribe 

e Ecosystem: wherein all members make more money 
or gain more prestige than had they not been part of 
the group 


Result: Allyship. Consumers are genuinely happy to do busi- 
ness with you, and root for your success; when you make a 
profit, they cheer, because they want you to thrive; they ad- 
vocate for you publicly> tying their personal brand with 
yours; they don’t even consider the competition; the old 
saying that “people buy from the person they like;” they 


would be OK with a small price-increase. 


4 Counter-example: Walmart and Amazon, known for exploiting 
workers and suppliers 


5 This single tweet demonstrates this with thousands of 
responses. 


Utility 


e Cheap: even if quality and functionality is low, it’s 
better than not having the product 

e Integrations: providing functionality while also more 
difficult to switch vendors 

e System-of-record: being the official place for 
important data, making it risky and expensive to 
switch vendors 

¢ Training: invested in having trained an organization, 


making it expensive and disruptive to switch vendors 
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Market-share Leader: the social-proof of selecting the 
market-leader is a reason to buy 

Location: coffee inside the airport is more expensive 
than on the street corner 

Convenience: groceries delivered to your doorstep 
are substantially more expensive than getting them 
yourself 

Simplicity: surprising ease is as delightful as it is 
useful 

Quality: a seamless experience with no defects is 
often worth paying for 

Risk-reduction: mitigating potential problems is 
difficult to measure, but valuable 

Unique functionality: a capability that no competitor 
can match is a sensible criterium for a purchase 
decision 

On-boarding experience: data shows that ease and 
reciprocity results in higher WTP 

Familiarity: having used a product or a workflow 


paradigm for years, it is the comfortable way to work 


Result: Fair exchange of value. Your product is useful and 


not excessively painful; the “devil we know;” getting your 


e Scale Anti-Pricing: raising prices once an installation 
is at-scale, knowing that although an alternative 
might be more effective, more desirable, and 
cheaper, the one-time cost of switching is incredibly 
high 

e Predatory Pricing: using lower-than-cost pricing to 
destroy competitors and ward off investors (funded 
by another business unit like Amazon does or by VCs 
as companies like Uber did), then increase prices 
once the competitive market has been decimated and 
customers have no choice. 

e Patents: abusing a system meant to temporarily 
protect inventions to block normal competition. 

¢ Corporate policy: once a product is written into a 
company’s formal policy (site-wide license or the 
only approved vendor for some application), that 
product “wins” even if every user hates it 


¢ Government fiat or regulation” 


Result: Adversarial. Customers want to leave; they idly 
comment that they wish some new competitor would arrive 
and disrupt you; they hate seeing your charges on their bill; 
they do business with you only begrudgingly; they lobby 


their boss to switch vendors. 


money’s worth; easier to stay than to leave, and no particu- 


lar desire to leave. 
Coercion 


¢ Contract lock-in: retaining your business through 
paperwork rather than by choice 

e Data lock-in: retaining your business by holding your 
data hostage rather than by choice 

e Effective monopoly: being the only feasible option® 

e Effective price-fixing: breaking the so-called “free 
market” 

e Middle-man: placing yourself in the middle of a 
transaction, increasing consumer price while 
decreasing supplier’s profit” 

¢ Bundle-stuffing: combining many things the 
customer doesn’t want with the few they do want, to 
charge more in total® 


6 This can be constructed purposefully, e.g. Uber spending tens 
of billions of dollars subsidizing rides to drive rival taxi and 
ride-share services out of business, so that they are the only op- 
tion, and can raise prices, as they now have done. 


7 & classic example is the person who buys a bunch of tickets to 
a concert, then resells them at 10x the price after the concert is 
“sold out.” Here’s an even more appalling example. 


foe) 


There is also a positive version of bundling, in which the items 
are mostly things the customer does want, purchased at a dis- 
count over buying each item individually, possibly with some 
useful interoperability. 


9 Here Uber is an example of “love,” breaking the “coercive” 
stranglehold of taxi industry regulations. 
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“Do me a favor and hit her with some 
lower expectations first.” 


All of these things contribute equally to the economist’s 
definition of WTP: The customer is in fact paying, and 
might pay more if you raise prices. But strategically they 


are completely different. 


Effect on growth and competitive 
pressure 


Love creates inexpensive, non-linear growth, because 
your customers are your allies. 

You get repeat purchases, whether it’s a one-time rev- 
enue product or a loyal recurring-revenue customer. This 
creates growth with no additional marketing and sales 


costs. 


You get word-of-mouth advocacy. When someone asks 
what to buy on Twitter, your rabid fans answer the ques- 
tion. When there’s a review site, your product ranks num- 
ber one. When Customer Surplus is enormous, con- 
sumers reciprocate by selling new customers on your be- 
half!® Once again, this is growth without additional mar- 
keting and sales costs. Furthermore, the effect grows as 


your customer base grows: A non-linear effect. 


When a new competitor arrives, even when it is superior 
in features or price, your customers will stay, because 
they’re here for more than just the features and the price. 


This yields retention, which is another form of growth! 


There is a limit to this effect of course—at some point the 
product simply isn’t good enough—but it carries you 
through the vacillations of typical competitive one- 


upmanship. 


10 Hollow Knight is a high-quality indie game, made primarily by 
just three people. Released in 2017, people still make YouTube 
videos about it in 2023. The soundtrack has millions of listens 
on Spotify. Everyone says it’s far too cheap at $15. Plus you get 
4 expansion packs for free—something games normally charge 
for. Everyone repeats the story about how it’s just two guys 
plus one other guy who did the amazing music. Fans even 
begged them to charge more but they don’t—they’d rather be 
accessible, and people love them all more for it. The economist 
would say they should raise prices because they can. Yes they 
can, but it’s obvious that rabid fans generate millions of pur- 
chases, and that financial impact is so much larger than closing 
the WTP/Price gap to “demonstrate you have market power.“ 


11 Don’t believe me? Look at the growth curve of any startup that 
went from 7%/mo cancellation to 2%/mo. 


Utility helps grow existing customers, and is neutral- 
to-positive on attracting new ones. 

As an organization grows, it will naturally buy more seats 
of software for teams in customer support, sales, engi- 
neering, and so on. It will naturally buy more in- 
frastructure and incur more credit card transaction fees. 
This isn’t a negative, and does creates internal growth, 
which is a powerful growth-driver for any business, espe- 


cially at scale!” 


But, a customer’s willingness to buy another ten seats of 
JIRA doesn’t imply the customer is going on forums, 
spending personal credibility to advocate on your behalf. 


And it doesn’t mean they won’t take a look at an interest- 


ing new competitor !3 


12 at scale, new customers can be added only so quickly, whereas 
you have an enormous existing customer base, so growth in- 
side the base is a larger factor than growth from new 
customers. 


13 Indeed, new JIRA competitor Linear has quickly amassed a ra- 
bid fanbase on the basis of exceptional UX. It’s easy to imagine 
JIRA users trying Linear and even advocating to switch, where- 
as it’s laughable to imagine a Linear user trying to convince 
their team to switch to JIRA. 
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Coercion causes your customers to be allied with 
competitors; they’re internally-motived to leave, so 
they will. 

‘Just give me an excuse.” Your customers, locked in 
against their will, cannot wait for a viable competitor to 
appear. They will go out of their way to switch, coming 
up with reasons why investing in the switch will pay for 
itself ten-fold, despite the cost. Exactly the case you don’t 


want your customers building. 


When your contract is up for renewal, you should be very 
afraid. When someone asks on Twitter what tool they 
should use, your customers say: “Well we use X, but don’t 


make the same mistake!” 


You are constantly vulnerable to disruption, even by 
mediocre competition. This is the weakest position you 
could be in, because you’re coercing customers instead of 


delighting them. 


Pete ceten—e 


“OK, little higher next time. You just gave him 
a hankering.” 


NG 
s Sy 


————— 


Profit done right: Create more positive 
WTP then split it with the customer 


“TWhen you increase WTP], youre adding value for the 
consumer, and then figuring out how to split that with 
the consumer.” 

—Michael Mauboussin, interviewed on the Invest like the 


Best podcast. 


Creating value for the customer comes first. Then—and 
only then—you can decide how to “split it with the cus- 
tomer,” either leveraging Consumer Surplus for advocacy, 


high-retention, and growth, or indeed by raising prices. 


When you create that value through Love or Utility, this 
is both sustainable and profitable. When it’s through 


Coercion, it is temporary at best. 


The strongest organizations have all three. For example, 
Apple generates Love through appealing design, being a 
statement of personal brand, and maintaining the highest 
standard of privacy even if it means the product is less 
functional or interoperable. Apple also leverages utility, 
becoming familiar and convenient (and thus a mental ef- 
fort to switch away), and trying to become the center of 
everything from family photos to shared files to 10,000 
notes to the common way to purchase things, creating a 
form of “lock-in” that feels useful rather than evil. But it 
leverages Coercion as well, as users are locked-in even 
when they’d prefer not to be, unable to export data from 
apps like Notes, and being forced to buy new devices as 
older ones suspiciously stop working well after applying 
new (mandatory) operating system upgrades, and chang- 


ing the connectors on charging cables every 5-10 years. 


In any case, Apple has increased WTP in all three ways 
over the past ten years, and they’ve split that with their 


customers, as evidenced by a consistent profit margin: 


Net Margin 


2010 2012 2014 2016 2018 2020 2022 


Apple’s net profit margin: 
If they were increasing prices faster than WTP, profits would 
have increased. 


6 of 9 


Willingness-to-pay: Creating permanent competitive advantage, for the right reasons 


Even the cold-blooded capitalist should 
eschew Coercion 


Here’s why Love and Utility results in more valuable 
companies, even though it prioritizes Consumer Surplus 
over profits: 


Imagine there are two companies, alike in every way: 
Same product, same industry, same market, same num- 
ber of customers at the same price, at the same costs, and 
thus the same revenue, same profit, and same WTP The 


only difference is: 


1. Company’s WTP is generated only by Love. 
2. Company’s WTP is generated only by Coercion. 


Which one is most likely to grow in volume and profit 
over the next five years? Which is more likely to capture 
more market share? In other words, which is the better 
investment for a Venture Capitalist? 


I'd pick (1). I know their customer base will help them 
grow efficiently, while competitors look on helplessly, un- 
able to convert customers even with the lure of unique 
features and lower prices. Whereas I know (2)’s customer 
base will be trying to leave, praying another competitor 
comes to save them, publicly warding away potential cus- 


tomers from repeating their mistake. 


It is also possible for (1) to add Utility or even Coercive 
WTP to their strategy, further strengthening their posi- 
tion, whereas it is much more difficult for (2) to generate 
Love starting from their current position. It’s not that 
Coercion is never an appropriate ingredient, but rather 


that the other two are better ‘+ 


14 Ts like the Agile Manifesto: When it says “Working software 
over comprehensive documentation,” it isn’t saying “Documen- 
tation is bad.” Rather, it’s saying “Working software” is more 
valuable, so that’s where we should spend most of our energy. 


Love beats Coercion, even as cold-blooded, money-grub- 
bing capitalist investor, indifferent to ethics or the better- 


ment of the world. 


And yet Love makes money while in fact bettering the 


world, and making everyone happier. 


So choose Love by building it into your strategy, invest- 


ing in it, and then reveling in what you’ve created. 


Appendix: Relationship to other 
frameworks 


You can apply this concept directly to your strategy, and 
merge it with other techniques. 


Kano 

“Love” feels a lot like Kano’s “Delight”—a joyous, perhaps 
even unexpected upside. ‘Utility’ maps _ to 
“Performance”—where the more of it there is, the more 
value it is to the customer “Coercive”? maps to 
“Inverse”—something that customers actively dislike, 
even though you gain the selfish corporate benefit of 


retention. 


Attractive 


Satisfaction 


Performance 


Functionality 
— —— 


Reverse 


Moats 

Many of these things sound like moats, and for good rea- 
son: Increasing WTP of any type increases your ability to 
capture and retain customers. The more forceful 


(whether positive or negative), the more that becomes a 
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permanent advantage that others cannot dislodge. No 
one can take away a fantastic brand, and government fiat 


can last for decades. 


An interesting example is “network effect,” because it 


shows up in all three types: 


¢ “Love” network effects include community and 
ecosystem, where participants help one another 
personally and professionally. 

e One “Utility” network effect is a functioning 
marketplace, so e.g. eBay was for decades the 
destination having the greater number of buyers and 
sellers of collectable objects, and thus genuinely the 
most useful place to transact. You might not “love” 
eBay, but certainly people went there because it was 
useful, not because they were forced to. 

e One “Coercive” network effect is when choice is 
limited to “preferred vendors,” creating a cartel 
rather than creating choice. For example, the United 
States health care system features insurance 
companies who each support their own network of 
doctors. A consequence is that switching insurance 
can mean you have to switch family doctors—an 
unnecessary and “value-destroying” activity as an 


economist would say. 


Start with “Why” 

“Love” reinforces the Simon Sinek’s admonition that 
companies must “Start with Why,” i.e. understand and ar- 
ticulate its higher purpose, it’s mission, because when 
that’s strong and important, when it permeates every- 
thing from its market-positioning to its culture to its em- 
ployees, it’s extremely powerful, and impossible for a 


competitor to destroy. 


Example: Buffer has a relatively undifferentiated product 
and pays lower salaries than many people can get else- 
where, but their culture and transparency is second-to- 
none, and people want to be a part of that. Example: 
TruthSocial, which can’t pay salaries like Twitter, and 
doesn’t have the reach of Twitter, and has technical issues 


with downtime and slow innovation, nevertheless pos- 
sesses a rabid fanbase because of the mission and 


community. 


Blue Ocean Strategy: The six kinds of “buyer utility” 

In Blue Ocean Strategy, W. Chan Kim and Renée 
Mauborgne highlight six ways in which you can deliver 
“value” to customers. These are a subset of the more gen- 
eral reasons why people are compelled to buy, but it’s 
useful to emphasize the cases where the customer is ben- 


efiting directly: 


Blue Ocean WTP 
Buyer Utility Category Commentary 
Customer Utility This category is too broad; it is 
Productivity important to distinguish between 
“more value” and “less cost.” Both 
contribute to “productivity,” but it 
is an order of magnitude more im- 
portant to increase value. It’s also 
important to define value. 
Simplicity Utility Included above. 
Convenience Utility Included above, in several forms; 
for example “location” is a specific 
kind of convenience. 

Risk Utility Included above. 
Fun & Image _—_ Love Included above. 
Environmental Love Included in a more expansive “so- 
Friendliness cial and environmental impact,” 
as nowadays (2023) it is more 
common for customers to make 
buying decisions on factors like 

Fairtrade, or purchasing from lo- 
cal or minority-owned business, 
or supporting businesses with spe- 
cific values and public commit- 
ments, in addition to the idea of 


being friendly to the environment. 


Long-term engagement metrics 

Many products wish to “drive engagement.” Some point 
to Facebook as the pinnacle of “growth hacking,” driving 
up numbers, often slipping away from Utility (to say 


nothing of Love) and into Coercive tricks. 


8 of 9 


Willingness-to-pay: Creating permanent competitive advantage, for the right reasons 


But even at Facebook, solving for Utility over Coercion 
worked better. In a fascinating multi-year UX experiment, 
Facebook found that when they reduced the quantity of 
notifications (by keeping the quality high), it had the ex- 
pected negative result on engagement: Customer satisfac- 
tion increased, but app usage decreased (because it was 
leading you back to the app less often). But, after a year, 
app usage actually increased and remained higher that it 


was before the change. Increasing genuine satisfaction 


created more engagement in the long run. They had to 


be patient to see the results; traditional “growth hacking’ 


did not discover the best solution. 


Many thanks to John Doherty for contributing insights to early drafts. 


Printed from: A Smart Bear 
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Using the Needs Stack for competitive strategy 


by Jason Cohen on June 4, 2023 


This simple method positions your product to be more 
valuable, especially against competitors who aim to disrupt 
you, or you them. 


“I knew it was bad when I stopped wanting a cracker 
and started needing one.” 


The Needs Stack 


How would you describe what is Charlie doing? 


Charlie creates an account with AWS—Amazon Web 
Services: pay by-the-hour and by-the-gigabyte for inter- 
net servers, storage, and network connectivity—happy to 
see a “free tier” for the infrastructure they need, just like 
the “how-to” article explained. All the acronyms are new 
and unfamiliar; this is tricky, but exciting! Someday, 
Charlie’s website might grow large enough to exceed the 


free tier—let’s hope! 
Apparently, what Charlie is doing is: 


* Buy infrastructure: web server, disk space, net- 
working, database. 


Nominally this is true, but Charlie’s life ambition is not 
“to buy infrastructure.” In fact, it’s a means to and end. 
What Charlie really wants is to have a functioning 
WordPress-based website, because WordPress is by an or- 


der of magnitude the most popular! way for people to 


easily write and publish content, and integrate with on- 
line marketing software needed to build a successful 


website. 


1 43% of the largest 10,000,000 websites use WordPress; Shopify 
is the next largest with 4%. (source) 


So, a better way to explain what’s happening, is that 
Charlie has a higher-level “need,” which is to set up a 
WordPress site. Getting the infrastructure is a means to 
an end, rather than the end itself; in this sense, it’s a 


“need” but it’s lower in the “stack.” 
What Charlie is doing really is: 


* Set up a WordPress site. 
— Buy infrastructure: web server, disk space, net- 


working, database. 


This might sound like a verbose restatement of Jobs to be 
Done or Five Why’s, but there’s a strategic competitive 
insight lurking here, which companies often fail to ap- 


preciate, at their peril: 


Charlie never wanted to set up infrastructure, and still 
doesn’t. Charlie wants a WordPress site. Therefore, a 
company who provides the higher-level need of “Word- 
Press site” makes the lower-level need of “infrastructure” 


obsolete. 


Specifically, when a company like WP Engine makes it 
possible to set up a WordPress site in one minute, with 
no infrastructure to think about, handling all the technol- 
ogy and 24/7 monitoring, and customer support that 
helps with WordPress, then that company is going to win 


Charlie’s business, and AWS will never see Charlie. 


Charlie disappears from the cloud infrastructure mar- 


ket. This is the insight. 
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If you ask AWS who their competitors are, the answer are 
other cloud infrastructure suppliers like GCP (Google), 
Azure (Microsoft), and Digital Ocean. And they are cor- 
rect, because the Charlies of the world who do buy in- 
frastructure, will buy from one of those. But WP Engine 
is also a competitor of AWS when it comes to Charlie, not 
because WP Engine competes on infrastructure (which it 
doesn’t, and in fact WP Engine itself buys infrastructure 
from AWS, GCP and Azure!), but because WP Engine tar- 


gets a higher-level “need” in the stack. 


But you can’t stop there. That’s the trouble—you always 
want to stop when you’ve reached the level you operate 


on, but there’s always another level. 


Instagram’s real product isn’t 
photos; it’s likes.” 


—Alex Danco 


It’s not Charlie’s life ambition to make a WordPress site; 
this is just a means to an end. What Charlie really wants 
is to have a personal website for content and self-promo- 
tion. Charlie wants to carve out a digital pied a terre. In 
the virtual universe there’s infinite terre, so why not slap 


your pied on some of it! 
Now the Needs Stack becomes: 


%* Have a personal website for content and self- 
promotion. 

— Set up a WordPress site. 

— Buy infrastructure: web server, disk space, net- 
working, database. 


Charlie can still achieve this higher-level need with 
WordPress running on WP Engine—indeed, over one mil- 
lion websites run this way today. But there are alterna- 
tives that target this higher-level need. Wix and 
Squarespace both let you create websites without 
WordPress, cutting out the step of “get WordPress” just as 


WP Engine cut out the step of “get infrastructure.” 


Does this mean WP Engine no longer has a business? No, 
that’s not the conclusion, any more than you should con- 
clude that AWS no longer has a business. WP Engine is 
13 years old and still healthy, growing, and profitable; 
AWS just posted $5.2B in profit just last quarter. So 


what’s going on? 


These alternatives to WordPress certainly have advan- 
tages, but they are worse than WordPress for long-form 
content and for customizability. For this reason, they 
might not be right for Charlie. But they are competitors. 
Similarly, WP Engine doesn’t let you customize your in- 
frastructure, so those who need to do so, still need to buy 
directly from AWS. 


The pattern here is the next insight: Up the stack, cus- 
tomers achieve their end-goal faster; down the stack, 


customers have more flexibility and customization. 


But wait... once again “having a personal website” is not 
Charlie’s life ambition, but rather just a means to an end. 
Charlie really wants a a book deal. Being a properly-pub- 
lished author with a hard-back volume on the shelf that 
you can show to your kids (who will just shrug today but 
maybe someday they'll appreciate it), and that you can 
gaze upon now and then in your dotage as a great ac- 
complishment. This is Charlie’s personal idea of “success- 
ful and popular.” The way to get a book deal is to already 
have an online following; you’ll have honed your writing 
craft, you’ll have a natural audience to kick-start sales, 
and the fact that you’ve won followers in the competitive, 
noisy Internet proves your content is worth reading. 
Publishers want authors whose content has been de- 


risked and who come with a built-in marketing channel. 
So the Needs Stack deepens: 


% Become a popular content-producer, leading to a 
book deal. 

— Have a personal website for content and self- 
promotion. 

— Set up a WordPress site. 

— Buy infrastructure: web server, disk space, net- 


working, database. 


2 of 7 


Using the Needs Stack for competitive strategy 


The new layer brings new alternative solutions. There are 
companies like Substack, purpose-built to help you gain a 
following, with the tools for long-form content, for build- 
ing a mailing list, and for promoting your content on so- 


cial media. 


Substack doesn’t automatically win, because WordPress is 
better than Substack in many ways. For example, 
WordPress is far more configurable, so you can make it 
look unique, unlike a million newsletters at Substack that 
all look identical (with character-kerning that’s too wide 
for my taste, but I’m biased). Also, the tools for long-form 
writing in WordPress are far superior, with drag-and-drop 
elements supporting complex layouts—including differ- 
ent layouts for phones versus tablets versus laptops—that 
an email newsletter cannot support. If Charlie values 
those things, then WordPress is still the right choice. 


This again exemplifies the pattern that software at higher 
levels of the Needs Stack have fewer features and less 
customization than software lower on the Stack. In ex- 
change for these limitations, their focus on the higher- 
level Need means it’s easier for the customer to achieve 
that Need. If the ease of achieving the Need is more im- 
portant to a customer than the ability to customize and 
extend, the higher-level product makes all lower levels 
obsolete. 


But wait a minute Substack because—you guessed it— 
there’s another level to Charlie’s Needs Stack. Because 
while Charlie wants a book deal as a signifier of populari- 
ty and success, it’s not Charlie’s life ambition. Charlie re- 
ally wants to become a famous speaker. It started at a 
Tony Robbins show; Charlie instantly knew that the stage 
was their calling. It’s the prestige of being “that person” 
at parties, and the idea of being a jet-setter in first-class 
and a black car to pick you up with your name in the 
window, the ego basking in the glow of the lights and the 
elevation of the stage, gazed upon by an adoring audi- 
ence who rushes up afterwards to grab a selfie and ner- 
vously blurt out a fraction of a story about how the 


speaker changed their life. 


And the money. 


* Get lucrative speaking gigs around the world. 

— Become a popular content-producer, leading to a 
book deal. 

— Have a personal website for content and self- 
promotion. 

— Set up a WordPress site. 

— Buy infrastructure: web server, disk space, net- 


working, database. 


Charlie’s plan isn’t bad, but what if it were possible to 
skip the website and years of writing and self-promotion 
and the book deal and the two years of labor creating the 
book and directly become the next Tony Robbins? It’s not 
impossible; there are speaker bureaus and online services 
like SpeakerMatch that help you find gigs and facilitate 
the transaction. Why spend years building an online fol- 
lowing (if you’re lucky) when you could spend that time 
creating inspirational presentations, delivering them to 
acclaim, earning testimonials for your next gig, building 
a reputation with an agency, and getting bigger and bet- 
ter gigs over time. In this scenario, Substack, 
Squarespace, WP Engine, and AWS are all irrelevant for 
Charlie; with the higher need met, the rest might as well 


not exist. 


The moral of the story is: You need to understand the 
Needs Stack of your target customers. Not just your 
“market” or your “product” or “solving the problem” or 
even the immediate Jobs-To-Be-Done of your customer— 


those are all means to ends. 


Already you can see how this will help you think about 
positioning and features; let’s see what else it helps you 
to do. 


How to leverage the Needs Stack 


The solution to “selling benefits” vs “selling features” 
“Sell benefits, not features” marketers have told us since 
the universe cooled enough for galaxies to form. We’re 
supposed to say “saves time” rather than showing a 
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screenshot of the feature. (Or better yet, “grows rev- 
enue.”) For Charlie, say “Become famous” rather than 


“Get a website.” 


And yet, does an Engineer looking for a database in- 
frastructure solution really want to be told “saves time” 
or do they want to see specs and features? (A: Specs and 


features.) 


If a construction worker is looking for a wrench, do they 
search Google for “wrench that saves me time” or “socket 
wrench for a 5/16” lock nut?" And when that person 
clicks a link to potentially buy the wrench, do they want 
pictures of happy people “saving time,” or do they want 
to see the specifications and features like “has a mounted 
light.” Do they really need to be told “the light is useful 
to see in dark corners?” Oh! Thanks for spelling out the 


benefits of photons! 


So which is it? 


“This is Tom. He creates awareness.” 


The Needs Stack suggests an answer. Charlie is not con- 
stantly and consciously thinking “I gotta become Tony 
Robbins,” even though that’s the top of their Needs Stack. 
Charlie is thinking at some other level of the stack at any 
given moment, for example “I gotta get some content 


online.” 


Google searches are a good proxy for “what level of the 
stack are potential customers operating at right now?” 
When Charlie embarked on this project, the search might 


have been something like “best way to publish online,” 


not “how to become a famous globe-trotting speaker.” 
After reading some articles about how WordPress is inex- 
pensive, popular and has all the tools you need, maybe 
then Charlie moved down the Stack to search “how to get 


a WordPress site.” 


If someone has in their head “get a WordPress site,” you 
want to meet them where they are, and talk about how 
your product is the best way to “get a WordPress site.” 


That’s the part where you talk about features. 


However, benefits still have a role to play, because it’s 
true that these features are a “means to an end,” and the 
more you convince the customer that you’re the best 
means to that end, you win even over alternatives. So, 
you can sell the benefits that are one level higher on 
the stack. 


In this case, you could imagine a home page message 
like: “Become famous online with our turn-key, cus- 
tomizable WordPress website.” That’s the benefit from 
one level above, promised as a consequence of the cur- 


rent level. 


Conversely, if you’re Substack, you can’t say “cus- 
tomizable” because that’s not what the product is, and 
it’s barely even a “website” (it’s just back-issues of your 
newsletter), but you can speak to the product “getting 
eyeballs.” So, that message could be “Grow your own au- 


dience with our turn-key newsletter platform.” 


2 Sure enough, as of this writing the Substack homepage says: 
“Substack lets independent writers and podcasters publish di- 
rectly to their audience and get paid through subscriptions,” 
which are benefits from one level higher (i.e. “be independent” 
and “get paid for writing”). Lower on the page it lists features 
from the level they are on: “A Substack combines a blog, news- 
letter, payment system, and customer support team—all inte- 
grated seamlessly with a simple interface. We handle the ad- 
min, billing, and tech.“ 


Position lower levels as irrelevant 

If Substack tried to fight the battle at the “get a website” 
level of the stack, they would lose every time—it has al- 
most no features of a modern website. But, it can fight 


the battle at the “build an audience” level; for some sub- 
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set of the market, that’s a more relevant level. Therefore, 
the idea is to make all levels below that irrelevant, by fo- 


cusing only on your benefits: 


“While others just give you a website, we help you build 
a following.” Right, because “popularity” is what I want- 


'” 


ed, not “a website!” This message won’t work on busi- 
nesses, for example, because they actually do need a 
website. But for the aspirational speaker, this might be 


perfect. 


Conversely, WP Engine does the same thing with cloud 
services. The major cloud services say: “Oh it’s easy to 
have a WordPress site. You just click 17 times on all this 
stuff to make a server, and then download and install a 
bunch of stuff to create server software, then set up and 
configure WordPress, then use this 8-page tutorial to in- 
tegrate with a fast caching network (which costs extra), 
then download and configure WordPress plugins to talk 
to the caching network, then stay on top of all the securi- 
ty patches and software upgrades, and it wouldn’t hurt 
you to learn how to SSH into your Ubuntu server so 


that....” 


Whereas, the Managed WordPress Platform says: “You 
don’t want infrastructure, you want a WordPress site, and 
in less than one minute, you'll have exactly that. And it’s 
faster and more secure than doing it yourself. And when 
you have trouble, you can call our support line, which 


cloud services don’t have.” 


In short, make “infrastructure” irrelevant by focusing on 
the next level up. The customer never wanted to think 
about infrastructure in the first place. 


Add more value by moving up the hierarchy 

If products targeted at one level disrupt the products be- 
low them, there’s an obvious strategic conclusion: You 
could disrupt yourself and your competitors by moving 


up the hierarchy. 


“Disrupting the marketplace sounds like a lot of work. 
Couldn't we just make a big mess instead?” 


There are huge barriers to accomplishing this. The barri- 


ers are so large, companies almost never overcome them: 


The next level is a different product, and a differ- 
ent business. 

If you’re known for selling WordPress sites, you’d have 
to re-brand to be known for building an audience, 
whether that’s actually a new brand or expanding an 
existing brand. You also need to become expert in 
what that product is, and build it? You might need to 
change your business model, e.g. selling infrastructure 
by the gigabyte, versus selling whole websites by the 
month, versus selling new subscribers on a pay-for- 
performance basis, versus taking a percentage of 


speaking fees. 


3 Or buy it, which mature companies often do, trading cash for 
speed-to-market. 


The next level isn’t possible 

It’s not clear that one can build a product that fulfills 
the promise to get someone a substantial following. 
Twitter and Facebook sort of can, in that you can pay 
them to advertise your content. But the Internet is a 
large and noisy place; marketing software doesn’t au- 
tomatically generate attention. Once you get the at- 
tention, if your content isn’t compelling, software can’t 


force people to subscribe. In short, if you go high 
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enough in the Needs Stack, you might run into some- 
thing that a product cannot produce. (Or at least you 


can’t.) 


The next level targets a smaller market 

Few people share Charlie’s Needs Stack exactly. A 
business with a broad customer base like WP Engine 
sells an all-purpose platform for all kinds of sites. 
There are many different personas whose Needs Stack 
includes “set up a WordPress site” In each case, the 
“next level up” is different. Therefore, targeting a 
higher level also means narrowing the market. This 
might be wise—it is a tried-and-true strategy to be the 
best product in a smaller niche, rather than a small 
fish in a large, crowded ocean. But, a company who 
has an established brand in the wider market might 
not want to narrow into a sub-market. There might 


not be enough money in it. 


Fag. individuals wanting attention, companies broadcasting 
their brand, media companies attracting eyeballs for advertis- 
ing revenue, eCommerce stores using content-marketing to 
drive sales, communities using content to inform and engage, 
non-profits raising money, governments interacting with 
citizens 


Since a full pivot is unlikely to succeed, there are other 
ways to build a strategy that partially climbs the 


hierarchy: 


Create sub-brands / products 

Keep the original business going, and launch “vertical” 
or “niche” products. Leverage the scale and opera- 
tional excellence of the parent, but treat each product 


as its own “startup.” 


Add features without fully committing to the next 
level 

Perhaps you can’t completely pivot to a product that 
promises subscriber-growth, but you could create an 
add-on product that helps someone do that. This be- 
comes an extension of an existing product, aimed at 
personas who share that next level of their Needs 
Stack. 


Better marketing 

Address the need with words instead of features. 
Without “promising” results, you can help, educate, 
and lead in that area. You can have a field guide about 
how to attract and retain subscribers to a blog. You 
can give customers ideas for new kinds of content. You 
could partner with other companies who help with 
promotion. You could highlight customers who have 
been successful, wherein they share their experience 


to help others. 


Still, while this advice is good for most organizations 
who cannot completely move up the hierarchy, there are 
examples of fantastic businesses that succeeded exactly 
because they successfully made the transition. For exam- 
ple, all of the case studies given in the “Emotional vs 
Functional” chapter of Blue Ocean Strategy, such as 
Cemex selling “build an addition to your house, for your 
family” instead of just selling cement or Starbucks selling 


a lifestyle experience rather than B-grade coffee. 


Measure the next level up 

The perennial question: Which few key metrics measure 
whether the customer is deriving value from our product, 
and thus predict whether they'll stay a customer? And be- 
yond that, become a vocal advocate? The Needs Stack 


helps us uncover these more strategic metrics. 


Metrics from the bottom of the Stack measure whether 
we're operating well, and certainly this is necessary even 
if not sufficient. Metrics at the level the customer is ac- 
tively thinking about are useful to measure whether our 
Product is delivering on its direct promise, which again is 


vital. 


It’s strategic to measure further up the stack. Not too far, 
otherwise it’s not only difficult to measure, but too many 
external factors drive the number. But the very next level 
up could be a critical way to measure value and to under- 


stand whether customers will be happy in the long run. 


If a customer of Shopify successfully creates a store, the 
product of “have a store online” is succeeding, and that 


will be reflected in metrics like “number of products cur- 
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rently for sale” and in user surveys like “how easy was it 
to set up products for sale.” That’s important, but what if 
no one buys their wares? That customer will probably 
churn, even though the software delivered on its prom- 
ise. Indeed, Shopify has enormous cancellation rates that 
would sink most companies; they survive because the 
ones that stay, grow a lot. Because they have so many in- 
bound new customers, they can manage around the fact 


that only 34% stick around for even one year. 


Can Shopify ensure that all customers who create a store 
end up doing thousands of dollars a month in business? 
No, this is an example where the next level up is vital, 
but Shopify cannot control it. However, they can measure 
it, and help. They could have articles about how to do 
online marketing, or features inside the store that help 
potential customers complete a checkout rather than 


abandon® 


5 Features like ShopPay, in which previous customers of other 
Shopify stores are fast-tracked through the purchase process, 
increasing the percentage of shopping carts that result in com- 
pleted transactions. 


This is why Shopify’s primary success metric is GMv°; 
they report it to Wall Street directly after revenue, and in 
the same large font. GMV measures whether their cus- 
tomers are succeeding in the next level up on the stack. 
GMV not only reflects the truth about where the cus- 
tomer value is, it also encourages their product mangers 
to think about how to stretch up from the level of the 
stack they fully control. 


6 Gross Merchandise Value: the total amount of money their cus- 
tomers are selling through their stores. 


A source of higher purpose 

Our direct goals are often simple: Sign up more cus- 
tomers, get them to activate on the product, reduce 
churn, reduce support tickets, reduce costs. All of that is 


important, but none of it creates a higher purpose. 
What’s it all for? Why should we even bother? 


Many companies can’t answer the question. Indeed, 
maybe they have no purpose other than “growth.” Will 
folks be motivated to do their best work at a company 


that values only conversion rates? 


Purpose can be found in the higher levels of the Needs 
Stack. The immediate work is “getting a website,” but the 
higher purpose is to enable someone to build a career for 
themselves, to engage their creativity and speciality, to 
help them win the respect of others. Or help them be- 
come a hero within a bigger organization through some 
new marketing project, which will eventually lead to 
their promotion. Or help them raise awareness and mon- 
ey for an important cause. Or enable them to speak truth 
to power in a region of the world where such speech is 
forbidden. 


Tell those stories. On the website and_ internally. 
Celebrate with your team when this actually happens for 
a customer, because that is what it’s all about. Those sto- 


ries are why we should care. 


Identify those upper reaches of the Needs Stack, not only 
because it helps you build more valuable features, not 
only because it helps you build a better strategy, but be- 
cause it is the ultimate reason why any of this is worth 
doing in the first place. 


Printed from: A Smart Bear 
https://longform.asmartbear.com/needs-stack/ 
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The three kinds of leverage that anchor effective strategies 


by Jason Cohen on June 18, 2023 


Leveraging strengths — not “fixing weaknesses” —- is how to 
win. Better when differentiated. Best when durable. Here’s 
how to create leverage. 


“I dunno, what if we leveraged something.” 


“Leverage” means generating a large effect from a rela- 
tively small effort, created by riding tailwinds of natural 
abilities or hard-won assets, rather than fighting a battle 


for which you are ill-equipped. 


Whatever you can do, or your company can do, that 
causes others to shake their heads in amazement, won- 
dering how you accomplished so much in so little time 
and so few resources, is more than just a “strength.” It is 


leverage. 


Some say that great strategy requires unassailable, hyper- 
differentiated, completely unique moats. While this is ob- 
viously preferable, it is unwise to ignore the more imme- 


diate and accessible forms of leverage. 


Leveraging strengths is the only way to do great work. 
(Not “fixing weaknesses.”) 
Better yet, leveraging differentiated strengths means 
you beat the competition. 
Best is when that differentiation is durable over 


time. 


Leveraging strengths 


How do you beat Bobby Fischer? 
Play him at anything but chess” 
—wWarren Buffet 


Why is it so easy to explain why a company succeeded, in 


just a few words? 


Netflix pivoted to streaming. Snap had disappearing mes- 
sages in a uniquely simple interface. The iPhone had un- 
paralleled design. Tesla solved both performance and 
safety for EVs. Google’s PageRank algorithm was an or- 
der of magnitude better than anything else, so it beat the 
other thirteen major search companies that had a head- 
start. Twitter’s limit of 140 characters, apparently use- 
lessly trivial, created a uniquely addictive communica- 
tions pattern, and meant it could be used even on cheap 


devices with cheap cellular plans. 


The story is always about how the company was amazing 
at one or two specific things. The story isn’t about the 47 
complex details that all added up, and isn’t about a 28- 


page strategy document. 


The story is also never about how the company “shored 
up weaknesses.” Twitter had massive weaknesses, like 


constantly going off-line: 
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(<)>- CC) og) Ct) Gant nntp://ewitter.coms WY WCB Google Q) 
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_ Twitter is over capacity. 


Please wait a moment and try again. For more information, check out Twitter Status » 


©2010 Twitter AboutUs Contact Blog Status API Help Jobs TOS Privacy 


Twitter Fail Whale circa 2010 


Fundamental issues like downtime and security flaws do 
need to be solved, but that’s not a strategy, it’s an opera- 
tional necessity. Facebook famously made this point when 
they pioneered the catchy motto “Move fast and break 
things,” explaining that rapid innovation is more impor- 
tant than anything else: 


MOVE 
FAST AND 
BREAK 
THINGS 


Books were written extolling the benefits and wisdom of 


this admonition: 


Jonathan Taplin 


How Facebook, Google 


and Amazon have cornered 
culture and what it means 
for all of us 

T i gq 


Except, once you’re no longer a scrappy startup, it’s not 
smart at all. The Fail Whale is not OK. Not even for 
Facebook, who officially changed their motto more than 


a decade ago: 


No one wrote a book about the new motto. Even Zuck couldn’t 
bring himself to write out the word “infrastructure.” 


Facebook didn’t defeat MySpace because of stable in- 
frastructure or breaking things; they won because of 
their strengths in hypergrowth. Twitter continued to 
grow during the many years of Fail Whale because of its 


unique communications experience. 


Reversing weakness is hard, painful, likely to result in 
something merely neutral, not great, and is at high 


risk of failing completely. Leveraging our innate great- 
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ness is “how we will win,” and “how we will win” is the 


entire purpose of a strategy. 


This is why typically the best answer to “what tool/lan- 
?” is 


guage/framework/process should I use to do 
“the one you already know1” That’s the one that best 
leverages your time—your most limited resource. 


1 Slack was famously built in PHP a language derided at the time 
as passé. But PHP is what the founders knew, and customers 
don’t care either way. Facebook used PHP too; when they 
scaled and needed more performance, they invented their own 
version of PHP Expensive, yes, but companies that never get off 
the ground never earn the “good problems to have” of scaling. 


Occasionally you do need to correct a strategic weakness 
in order to win. Not an operational weakness like “unsta- 
ble infrastructure,” but one where you’re constantly los- 
ing deals to a competitor, due to a major deficiency in 
product or positioning. Something where we need to 
change “who we are” in order to win. 


A good strategy will explain why this risky, difficult, un- 
natural, expensive journey must nevertheless be under- 
taken, and why it’s the best way to win. There must not 
be many; probably not more than one. You can’t afford 


the time, people, or energy. 


Better is to avoid weaknesses through decisions on target 
ness is that you cannot supply 24/7 customer service like 
some competitors, call yourself a “boutique,” meaning a 
business where customers have incredible interactions 
with experts, but of course those experts are available 
only at limited times. If your weakness is a small team 
who can’t build millions of features, make a product tar- 
geting users who have simple needs and appreciate sim- 
ple, well-built software that “doesn’t have all that crap I 


don’t want.” 


“Flight, super strength, super speed, bullet proof, 
heat vision, freeze breath... And you’re upset 
because you can’t see through lead?” 


Some weaknesses can be debilitating if they cross some 
threshold, but otherwise they should be ignored; uptime 
is one of them. For these types, at WP Engine we’ve had 
great success with the system of SLO management from 
Google’s SRE Book”; while written about infrastructure, 
it’s a general-purpose system for managing things that 


should be satisfied rather than maximized. 


2 You define a metric that measures the effect in question (e.g. 
“uptime in past 28 days”), and you define the acceptable 
threshold (e.g. “99.5%”). The rule is: while the metric exceeds 
the threshold, we do not attempt to improve it—that’s like 
over-investing in a weakness that will never become a strength. 
But if the metric drops below the threshold, the team addresses 
it—that solves for the “minimum acceptable weakness.” By be- 
ing objective and explicit about when we invest, we invest just 
the right amount, at the right times. Imagine a downtime inci- 
dent, with emotions running and and demands that “we must 
invest so this never happens again.” That’s when you need a 
previously-agreed-upon decision-making rule. 


A cross-functional team meets quarterly to agree on the 
threshold; this way, when some downtime happens, and 
everyone is emotional and demanding that “something 
must be done, so this never happens again,” you can go 


back to your rule and point 


Leverage isn’t just about strengths; it can also come from 
the business model. A common business model amongst 
self-funded entrepreneurs is a product with near-zero 
marginal cost Software works this way, and also con- 
tent: blogs, newsletters, podcasts, videos, books, course- 
ware. A newsletter written for 100 people takes the same 


effort as for 100,000, yet the latter can generate 1000x 
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more revenue. This is why a self-funded a content com- 
pany is difficult at first (all effort, no income, no com- 
pounding effects) but can be extremely profitable if it 
takes off (far more revenue for the same effort and al- 


most the same costs). 


3 Meaning: Creating the first copy of the thing might be expen- 
sive, but selling the next copy of the thing costs almost nothing. 


Leverage can also come from supporting tools. As of this 
writing, AI can generate a simple blog post, albeit with 
factual errors, no personal examples, and a plodding 
style. But, AI can draft ten articles, and a human can edit 
those many times faster than generating them from 
scratch, resulting in a large increase in output. Similarly, 
a great strategy might mandate the creation of a specific 
supporting system that doesn’t just “make us more effi- 
cient,” but makes us 10x more efficient, and therefore be- 


comes a new form of leverage. 


Days before OpenAI Days after OpenAI 


Write content - 3 hours Write content - 5 minutes 


Edit & Fact-check content Edit & Fact-check content 
- 4 hours - 12 hours 


Gl (Gly 


Passion is leverage. Your strengths are magnified through 
sheer will, and that same energy helps you plow through 
the emotional pain and drudgery of working through 
challenges. This is why “follow your passion” is correct 


but incomplete. 


Magic is just someone 
spending more time on 
something than anyone 
else might reasonably 
expect.” 

—Raymond Joseph Teller (of 
Penn & Teller) 


Leveraging differentiated strengths 


The kinds of strengths we’ve just identified are good, but 
your competitors have those kinds of strengths too. 
(You’re not the only organization with “super-talented, 
passionate engineers,” and all content companies have a 


leveraged business model.) 


The more important strengths separate your organization 
from others. These are more strategic, because they help 
you win against the competition, not just by leveraging 


your time. 


It’s simple conceptually; these are the clichéd “differenti- 
ated competitive advantages” or “unique selling proposi- 
tions.” The simplicity is deceptive: it is difficult to invent 
differences which aren’t trivial, and that also apply to 
most of your target audience. This difficulty is why these 
need to be identified in the strategy; we won’t just hap- 
pen across these by accident during day-to-day work. 


Attack where you are strong and 
the enemy is weak; do not attack 
where the enemy is strong and 
you are weak.” 

—Sun Tzu, Art of War 


Technology and design choices can create competitive 
differentiation. A technical architecture will make some 
features and capabilities easy, and others difficult or im- 


possible* A design can make certain use-cases obvious 
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and delightful, while others are annoying or impossible. 
There is no such thing as technology or design that 
makes no trade-offs. Great strategy names those trade- 
offs, shows how they align with the target market and 
target persona, and therefore how you'll exploit those 
trade-offs to win, while competitors who made different 
trade-offs won’t be able to follow, at least not at extreme 
cost and taking far more time just to catch up to where 


you already are today. 


4 Heroku invented the now-ubiquitous idea of the 12-factor ap- 
plication, but only after mandating a combination of trade-offs 
which many Rails developers initially hated, like requiring 
Bundler for package management (back when Bundler was 
bad), mandating PostgreSQL instead of MySQL (back with the 
latter was 10x more common), and disallowing writing data to 
disk. Developers ultimately accepted these restrictions because 
taken together they enabled a new way to develop, test, stage, 
deploy, and scale applications. Today, nearly all SaaS ap- 
plications follow the 12-factor principles, and Heroku cleanly 
won the Rails market for a decade. 


un 


But also, how those trade-offs create limitations or problems 
for you, that you will need to accept. You must not call it a 
“bug” when those limitations appear. They are the consequence 
of leaning into your strengths, and trying to “fix” them will 
take inordinate effort, they will not result in a fantastic prod- 
uct, they blunt the power of your strengths, and they remove 
time and energy that could have been spent magnifying those 
strengths. 


A specific type of technical differentiation is solving a 
particularly difficult engineering problem® This has the 
advantage that, if you succeed, you might have created a 
moat (e.g. Google search, Tesla cars, OpenAI). The disad- 
vantage is that it is likely to fail: take too long, take too 
much money, not actually solve the difficult puzzle, not 
do enough of what the customer needs. Therefore, you 
should be surgical about which subset of the software 
should be difficult: Something that creates a competitive 


moat and that leverages some strengths or assets. 


6 Google Wave was supposed to change everything: Replacing 
chat, email, word-processing, spreadsheets, file-storage, note- 
taking, wikis, and websites. Built as the conceptual precursor 
to Google Docs, it was the first product with remarkably fast 
multi-user simultaneous editing; you could watch people type 
character-by-character. The technology was fantastically diffi- 
cult; no one else had it, and it would be years before anyone 
else did. Unfortunately it was so difficult that it failed, but this 
proved the rule that, had they succeeded, it would have creat- 
ed long-term differentiating leverage; project co-founder Lars 
Rasmussen said it was so difficult, that even if they were al- 
lowed to it all over again, with the same team, with all the 
knowledge they now possessed, it would again take years to 
execute, and still probably wouldn’t have worked. That means 
the competition would have failed too. 


Besides “big awesome features,” another kind of differen- 
tiated advantage is an insight—when you understand an 
important truth while the rest of the world does not, or 
especially if the rest of the world confidently believes the 
opposite to be true”. Their disbelief prevents competition. 
Once proved true, others will follow’ but it’s too late; 
the strategy has already been a success. Made famous as 
Peter Thiel’s “Secret,” these are rare but valuable insights 
that underpin some of the most famous and successful 


corporate strategies. 


7 “People will buy shoes over the internet.” [Zappos] // “People 
will take rides from strangers.” [Uber, Lyft] // “People want to 
sleep in stranger’s houses.” [AirBnB] // “Rockets can be re- 
usable.” [SpaceX] // “The highest-performing sedan can be 
battery-powered.” [Tesla] // “People communicating in 140- 
character chunks can topple governments.” [Twitter] // “Large 
enterprises will acquiesce to putting their customer data ‘in the 
cloud’ (a term we just invented).” [Salesforce] // “Open source 
copycats will defeat multi-billion-dollar incumbents.” [RedHat] 


8 Only after Salesforce convinced the world that SaaS is viable, 
nearly all software companies copied the model. Only after 
Linux defeated IBM, Sun, HP and everyone else, there’s now an 
open-source copycat of everything ever invented. Only after 
RedHat successfully monetized “free” open source software, 
reaching tens of billions of market-cap, did investors “open 
source companies” now resulting in hundreds of billions dollars 
of market capitalization. 


Unfortunately, differentiation tends to erode over time, 
sometimes rapidly. “Secret insights” make themselves 
known over time, and almost anything in software can be 
copied. Patents are supposed to provide protection, but 
what software company prevents copycats through 


patents? 
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Therefore, most valuable, strategic differentiation is of a 


kind which takes decades to erode, if ever. 


Leveraging durable differentiated 
strengths 


The most important thing to me is 
figuring out how big a moat there 
is around the business. What I 
love, of course, is a big castle and 
a big moat with piranhas and 
crocodiles.” 

—Warren Buffet 


The word for a durable differentiated advantage is a 
“moat.” Moats often take the form: “I have something 
that you don’t, and furthermore, you either don’t have 
the ability to acquire it, won’t invest enough to achieve it, 
or in the case that you try, you would have to make an 
incredible investment of time and energy and possibly 
brand adjustment to do it, which is unlikely to succeed, 
and even if it does succeed it will take years, by which 


time we’ve already won.” 


Visit this companion article about moats for examples, in- 


cluding a worked case-study. 


Besides those classic moats, a special and often-over- 
looked form of durable differentiation is to exploit an os- 
sified structure in a competitor, where they cannot react 
even if they wanted to. 


¢« A competitor with a high cost structure must charge 
high prices, therefore a strategy of have “40% of the 
features at 10% of the price” leverages a point of 
differentiation that the competitor cannot match. 
This is simplistic form of Innovator’s Dilemma 


disruption. 


e« Acompetitor with a simple product and no tech 
support won't be able to win complex clients with 
complex needs, therefore a strategy of “Enterprise 
offering at Enterprise prices” leverages differentiation 
that competitor cannot match. 

e A competitor who loudly, publicly espouses a set of 
values cannot take actions contrary to those values, 
therefore a strategy that espouses a contrary set of 
values will remain differentiated”. 

¢« A competitor who invests in a core platform with 
specific trade-offs will not be able to change those 
trade-offs for years, if ever; therefore selecting 
different trade-offs, resulting in different product 
features (and liabilities), will remain differentiated. 

e« Acompetitor whose profit is driven by a specific 
product cannot lower prices when another company 
decides to build that same product for free as a loss- 
leader that powers a different line of business: 

o Netscape (revenue from selling a browser) was 
killed by Microsoft with IE (loss-leader for 
Windows) 

o For-profit smartphone operating systems were 
killed by Google with Android? (loss-leader for 
the Google ecosystem of services) 

o Twenty proprietary Unix operating systems were 
killed by RedHat with Linux (loss-leader for 
RedHat’s Enterprise Services, later sold for $34B) 

© Quoting Microsoft CEO Satya Nadella on his 
strategic approach with Bing, “There is such 
margin in search, which for us is incremental. For 


Google it’s not, they have to defend it all.” 


9 A company who values sustainable sourcing cannot price-com- 
pete with those who lack that value; conversely a sustainable- 
sourcing-minded consumer will buy from the former despite 
the expense. 


10 Except Apple of course; still, 87% of global handsets are 
Android, and Apple arguably remained because of the hard- 
ware more than the operating system. 


Finally, in a crowded competitive space, it’s possible that 
no single thing is completely unique, much less durable. 
A combination of multiple things taken together can be 


both differentiated and durable, so long as the con- 
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stituents form a mutually-reinforcing, cohesive structure, 
in which a competitor who successfully copied a fraction 
of the structure will still not have created a similar 


product!} 


11 This idea is also explored in Blue Ocean Strategy, though it 
originated years earlier as in the example below. 


Consider the example of Southwest Airlines, a low-cost 
US airline with an incredible business model, remaining 
profitable even during the US terrorists attacks of 
September 2001 and throughout the Great Recession of 
2008, across a period where every other major US airline 


filed for bankruptcy at least once. 


Southwest doesn’t have any one thing that is completely 
unique in the industry, and certainly nothing that a com- 
petitor couldn’t copy if they wanted to, but it does have a 
combination of decisions, features, strengths, and even 
weaknesses, that together are unique, have been durable 
for sixty years, and explains their success. Michael Porter 
diagrammed this combination in 1996; his analysis is re- 
mains accurate more than twenty years later, underscor- 


ing its durability: 


Southwest Airlines’ Activity System 


Limited 
Passenger 
service 


Frequent, 
reliable gate r point-to-point 


departures reraft routes between 


midsize cities 
and secondary, 
airports 


ns 
Lean, highly D Very low 

productive ticket prices 
ground and 
gate crews 


High 
aircraft 
utilization 


Michael Porter’s “Activity Systems Map” for Southwest Airlines 


Notice how some attributes would be considered 
strengths by a customer (e.g. low ticket prices and fre- 


quent, reliable departures), while others are weaknesses 


(e.g. no meals, no baggage transfers, no connections to 
other airlines, no long-haul routes). The key is that all 
decisions support a common structure, e.g. the lack of 
amenities is part of what allows them to have lower 
prices, and short-hauls on standardized aircraft (which 
helps with cost-reduction) also leads to frequent, reliable 


departures (that customers love). 


Strategy is a set of interrelated 
and powerful choices that 
positions the organization to 


win. 
—Roger Martin 


It’s easy to claim that “our peculiar set of features makes 
us unique,” but that’s not strategic unless it forms a mu- 
tually-reinforcing network, including weaknesses. It’s not 
durable leverage unless a written strategy explains this 
network, so that the company can continue to reinforce 
the entire network with its actions and investments, in- 
cluding accepting the weaknesses, not treating them as 
“bugs to be fixed.” 


Otherwise, it’s just a pile of features. 


Give me a long enough lever and I 
shall move the world.” 


—Archimedes 


Without leveraging strengths (rather than spending far 
more energy shoring up a weakness that still won’t be 
great), the company will not succeed in creating some- 


thing great. 


Without leveraging differentiated strengths, the company 
will not surpass competitors, will have a hard time win- 


ning and keeping customers, and will have an even hard- 
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er time justifying profit-generating prices. 


Without leveraging durable, differentiated strengths, the 
company’s success will be short-lived, differentiation will 
be temporary, and once again it will be reduced to out- 
spending on marketing or lowering prices until it is 
unprofitable. 


A winning strategy explains which strengths the company 


will leverage, how it will side-step rather than “attack” its 
weaknesses, which strengths can be leveraged for differ- 
entiated sales today, and which long-term moats the com- 


pany is constructing. 


Printed from: A Smart Bear 
https://longform.asmartbear.com/leverage/ 
© 2007-2024 Jason Cohen ‘W @asmartbear 
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For probabilities, use Fermi numbers, not words 


by Jason Cohen on July 2, 2023 


Don’t use phrases like “unlikely” or “almost certainly.” Here’s 
real-world data showing why not, and what to do instead. 


Distribution of responses according 
to respondents’ estimate of likelihood 


Word or phrase 

Always J 
Certainly 

Slam dunk 
Almost certainly 


Almost always 
With high probability 


Usually ~~ N 
Likely aN 


Frequently —> 
Probably _— > 
Often —™ 
Serious possibility ai ecemaionlllia. 
More often than not p X 

Real possibility ncinseceeiieiiliaiiimis 


With moderate probability 


_ 
Maybe J \ 


Possibly ee 
Might happen i 
Not often ph 

Unlikely Lo 

With low probability 

Rarely 

Never 


0% 50 100 


Source: Andrew Mauboussin and Michael J. Mauboussin 


Words have fuzzy meanings, but that alone doesn’t mean 


words are useless. 


However, we find serious problems when we examine 


these words in more detail. Another study: 


0% 10% 20% 30% 60% 70% 80% 90% 100% 
Almost Certain K-—_ dt 
Highly Likely +++ {+ 
Very Good Chance ee ae oe | 
We Believe 
Likely 4 + -—_+_ i ++ { 
Probable é [+-++—_4{_ 4 ff + 4+ +++ 
Probably /_—__ fp +-—-—_ 
Better than Even ‘ +t: ° 


About Even e foc * 
ee 
eS 


Unlikely 4 


improbable f#-—-@ fo 8 |-*+—e-.—4 5 ® 
Chances are Slight $8s-8#*—{i np} —#—4 ® . ° 
Little Chance +#5"2§Mfp—#—+—} 8 ° 
Highly Unlikely #9¢2-{Eb—4 8 © & ° eo. 
Almost No Chance theo 4 3 ey 


Note that the median intended probability of the word 
“likely” is 70%, whereas the intention of “unlikely” is 
20%. Since that is the same word, just negated, we 
should expect the numbers to be symmetric like 70/30 or 
80/20, but instead we’re biased ‘ 


Even worse with “probable / improbable” at 70/10, and 
strangely while “probable” and “probably” are the same 
at 70%, “probably not” is 25% while “improbable” is 
10%. Perhaps that makes sense to a grammatician, but 
surely this causes confusion in normal people, especially 
when we're applying these words in analytical contexts 


like risk-analysis or debating a strategy. 


1 There is extensive literature on our innate bias regarding both 
rare and common events; people are surprised how often 5% 
things happen, and how often 95% don’t. Nassim Taleb’s Black 
Swan expounds on this, and it’s obvious in our everyday expe- 
rience, like how we get upset when a “10% chance of rain” 
rains out our picnic. 


The communication problem is even worse because indi- 
vidual people disagree to an even larger extent. For exam- 
ple, here is a single person’s evaluation for these words: 
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Probability Perception 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 
Almost Certain ~ +—— fH 
Highly Likely Very Good Chance was perceived to be 70%. +—1 
This some use repatss to ach neta Seledn the ialeaton 
Very Good Chance 2 ; 
We Believe 2 ° I 
Likely 4 ¢ I 
Probable 4 |». I | 
Probably °. | | 
Better than Even Kt ¢ 4 
About Even ¢ 
Probably Not. { ° | +} | 
Chances are Slight ++ 
Little Chance +-— fina 
Highly Unlikely {i ° 


Almost No Chance = 


Note how pessimistic they are on the positive words, 


29 66 


scoring “probable,” “probably,” and “likely” as only 50%. 
Then matching the median of 20% with their negatives 
(“probably not,” “we doubt,” “unlikely’), but then they 
claim “improbable” is a straight-up 0% while “highly un- 


likely” is still 30%. 


I feel for this person. Although I admit the formal defini- 
tion of “improbable” cannot be exactly 0%, what would I 
think in a real-life case, in which a consensus is that a 
given project is “improbable” to complete on-time? I 


would certainly act as though that probability were 0%. 


So this is the point: Words don’t work, because the differ- 
ences between individual interpretations are larger than the 


differences between the words themselves! 


Furthermore, even if you assign an official numeric prob- 
ability to words, and train competent people to adhere to 
those definitions, it still doesn’t work. The CIA attempted 
exactly this, publishing what they call the “Sherman Kent 
Scale.” It explains, for example, that “probable” is de- 
fined as 75% and “almost certainly not” is defined as 7%. 
However, when officers were assessed against the scale, 
fully half the time they failed to stay in-range (the grey 
bars): 


Statement 
Almost Certainly + omp $P og 
Highly Likely 4 000 @Po0 0% 0g 8 
Very Good Chance 4 000 @0 0% C804 to. 8 
Probable + grr 080 0 000 0 Bow 8B 
Likely 6,008 © 0 000 [6 eo WS o 
Probably + © 000 000 9 000° © Bo %w00 
We Believe + © 000 020 9 0 oe 0 © © Be %eo 
Better Than Even 4 $8309 
About Even + 


We Doubt +20 offe oof o Boo 
Improbable + © ae cof © Sooce 
Unlikely +°°-%6ta-@ > © 
Probably Not + @@@ Gs § © 
Little Chance + 8896-0 © go 
Almost No Chance $&%=<e 
Highly Unlikely | 8885e0 © 


Chances Are Slight /8e@es-0.° _@ ° 


1 T + t + T 1 r 1 
0) 10 20 30 40 50 60 70 80 90 100 
Assigned Probability (%) 


e Probability assignment by NATO Officers 
Probability ranges proposed by Sherman Kent 


“Clearly, the readers in this experiment were not using the 
Sherman Kent scale even though they were familiar with it." — 
Scott Barclay, author of this study 


We should give up on using words to describe 


probabilities. 


The solution: Specific probabilities, 
Fermi-style 


An obvious solution is to force people to always use nu- 
meric probabilities, never? using vague words. Indeed, 


this one rule will already improve communication. 
2 Oops, I mean 0.2% of the time. 


But writing down precise probabilities is hard. 


It’s hard because usually® we cannot specify the probabil- 
ity with precision. The world is often* unpredictable even 
with expertise and data, so we need fuzzy ranges of prob- 


ability to gesture towards our intent. 


3 Cops, I mean 83.6% of the time. 


4 Oops, I mean 62.9% of the time. 
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In this sense, it could appear more accurate to say “the 
project is unlikely to succeed” exactly because it’s un- 
known whether the true probability is 10% or 40%. Still, 
given individual interpretations of the word “unlikely,” 


we're not accurately communicating that range. 


Furthermore, as we know from Fermi Estimation in do- 
mains like “impact” and “time estimation,” it’s unproduc- 
tive to haggle over details like “is it a 20% or 30% proba- 
bility.” None of us likely? knows the true number, and 


anyway we need crisp signals to make smart decisions. 
5 Oops, I mean 9.1% of the time. 


Therefore, the solution is a Fermi-style probability: 


1. Use numbers, not words. 


2. Select from a small set of options. 


This is the same conclusion the CIA came to in the above- 
referenced 1964 study. Their mistake was to continue us- 


ing words, but their categorical probabilities were Fermi: 


Probability Word Probability Value Expected Range 
Certain 100% 100% 
Almost Certain 93% 87% ... 99% 
Probable 75% 63% ... 87% 
Chances About Even 50% 40% ... 60% 
Probably Not 30% 20% ... 40% 
Almost Certainly Not 7% 2% ... 12% 
Impossible 0% 0% 


My recommendation is to use just a few raw 
numbers, without words, with instructions for 


“rounding off” that depend on the context. 


For example, in estimating the likelihood that a project 
completes on time, we know that in general things are 
more likely® to be late than early, therefore we should 


“round off” towards the lower probability. 


6 Oops, I mean more than 51% of the time. 


“I know mathematically that A is more likely, 
but I gotta say, I feel like B wants it more.” 


So go to your “risk” slides, and use Fermi probabilities to 
force yourself to decide what you think the risk really is, 


so everyone can decide whether or not to act. 


Go to your strategy, and put Fermi probabilities on each 
assertion, so people know what is more or less likely to 
change as we learn and grow. 


Go to your work-prioritization system, and put Fermi 
probabilities on the “value” or “estimate” metrics that 
you’re using for decision-support. Ask everyone in the 
team to supply their own numbers independently; where 
there’s disagreement, that’s worth a discussion; where 
there’s agreement, you can save time by just moving 


along. 


Perhaps the real reason we use wishy-washy probability 
words is because we really don’t know the probability, 
and rather than admitting that, we just glide past the 


challenge. That’s the worst reason of all. 
Now you have no excuse. 
Be brave, and put a number on it. 

Printed from: A Smart Bear 


https://longform.asmartbear.com/probability-words/ 
© 2007-2023 Jason Cohen WW @asmartbear 
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Selecting the right product metrics (KPIs) 


by Jason Cohen on July 16, 2023 


A novel system for selecting and presenting product KPIs, 


satisfying not only the product team, but also stakeholders, 
executives, and customers. 


TO 


NM 


“Thompson, I need you to redefine key metrics through 
dynamic optimization alignment. The rest of you, 
figure out what that means.” 


Executives want financial outcomes, strategists want sys- 
temic impact, managers want team accountability, teams 
want credit for executing work, planners want to track 


progress, ops wants to know that systems are stable and 
secure. 


They’re all correct, so how do we select metrics that satis- 


fy everyone? 


Much blood has been spilled on this topic. Frameworks 
range from poster-sized interconnected networks of box- 
es_and arrows, to reductively selecting a single North 
Star metric to rule them all, to cascading goals like OKRs. 
An organization should pick whichever framework is 


most likely to be adopted and honored. 


I used to believe that “one true metric” with a smattering 
of operational indicators was the best way to focus a 
team on “what matters most.” But I’ve come to believe in 
a more comprehensive system, that addresses all of the 


needs outlined in the opening paragraph. 


Here is that system. Because all departments can “see 
themselves” in the result, I believe it makes stakeholders 
more comfortable, while giving the product team at the 
center of the maelstrom a practical view, not only of 
everything they do, but of everything they affect. It al- 
lows the team to measure things in the short-run while 


also maintaining the long view. 


Metrics as a (simplified) value-chain 


A product sits in the middle of a chain of events, execut- 
ed by the team, customers, and peers across all depart- 
ments. The first step in understanding metrics, is to plot 
these events in time, by actor, and by the type of so- 


called “value” we might measure: 
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Product KPI Framework 


Value Realized 


A (customers’ biz metrics, GMV, 
Higher-level time saved, JTBD being D) 
subsequent 
value 
o 
€ 2 Advocacy 
g 2 (referrals, reviews, tweets, 
no = i P reposts, case studies) 
6 XZ oDiirect Sales (Self-Serve) Activity Satisfaction 
value (new, upgrade, (usage, artifacts-created, (Service CSAT, NPS, 
take-rate) DAU/WAU/MAU) surveys) 
Financial 
(ARR, NRR, ARPU, [— 
LTV, retention) 
Directly Marketing Sales (Assisted) Progress Collaborative 
visible (visits, views, leads, (new, upgrade, (features, bug-fixes, data Progress 
by CPC, CAC) win rate, close-time) surfaced, documentation) (integrations, bundles) 
Customer 
2 > 
Cio 
Q8 Health 
€ o uptime, latency, scale, 
Ip 
8 <x MTTR, test coverage) 
Far from 
Customer 
Maintenance Costs 
(security patches, (infrastructure, incidents, 
library & API updates) internal training) 
Ye 
Upstream Direct Domain Downstream 
from Product Team of Product Team. of Product Team 
« Leading * Immediately measurable « Lagging 


* Value potential 

* Multi-factor 

* Product Team influences; 
doesn't fully control 


+ Value for the customer 


« Value for the company 


* Single-cause 
* Product team controls the outcome 


* Multi-factor 
« Product Team influences; 
doesn't fully control 


Defines Success 


| Maximize | | 


Satisfice | | Input 


Use color to specify which metrics you are maximizing vs 
satisficing. Often “maximizers” are critical for the success 
of the product, whereas “satisficers” are important opera- 
tional indicators that can’t be ignored, and require atten- 
tion if they become a problem}, but under normal cir- 
cumstances shouldn’t distract us from what is most 


important. 


Once you have this map, the metrics almost write them- 
selves. Each box has one or more metrics that explains 
whether “it is happening” or “we’re making progress.” 
You could go a step further, adding arrows? to indicate 
influences or funnels or conversions, adding metrics to 


each arrow. 


a 


At WP Engine we’re very happy with Google’s system for man- 
aging SLOs, in which “satisficing” KPIs are tracked continuous- 
ly, but the team acts only when a KPI slips into “violation” terri- 
tory, as opposed to fielding requests to invest in improving 
those KPIs. This threshold is pre-agreed with stakeholders, in 
periodic meetings outside of high-emotion catastrophes, when 
everyone can soberly decide under what circumstances we will 
interrupt high-value work to address a critical problem. 


2 Also called a systems diagram, it could make the chart too 
busy, especially when “many things affect many things,” result- 
ing in a dizzying bird’s nest of arrows. Perhaps the simplicity of 
“boxes only” outweighs the benefit of specifying all the value- 
flows. Or maybe include only the few, most-important arrows. 


The axis of “time, immediacy, and 
control” 


The horizontal axis emphasizes that some events happen 
prior to activity controlled by the product team, and 


some happen after. Most metrics frameworks have this 
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concept of “leading” and “lagging” indicators, though 


some confusingly mix them all together®. 


3 For example, often top-level OKRs are multi-input, lagging in- 
dicators, whereas team KPIs might be immediate, and it’s un- 
clear where leading indicators go. This leads to unhappy con- 
versations when teams meet their immediate KPIs, but the 
company’s overall KPIs appear unaffected. 


Between those temporal bookends, we highlight that 
some of a product team’s activity can (1) be measured 
immediately and (2) the team is in full control of those 
effects, and therefore the team should be held directly 


accountable to those things. 


This resolves the typical conflict that arises when execu- 
tives ask “why isn’t the team more focused on increasing 
revenue in the next 60 days” while the team insists “oth- 
er people don’t understand that we’re doing a lot of im- 
portant work.” Work can—and should—be measured 
sprint-by-sprint, whereas revenue is a multi-input, lag- 
ging indicator of success. The product team is responsible 
for generating revenue, but it is not the only team or ac- 
tor contributing to that final result, and a change in the 
product can take a while to show up in revenue; individ- 
ual features often cannot be directly linked to revenue at 
all. 


This doesn’t make “revenue” less important—indeed, it 
might be the most important metric! Rather, we have 
placed the metric in context, and understood that it can 
lag by months or even years‘, and therefore isn’t a good 
measure of what’s happening right now. If we’re success- 
ful at our “work” but not our “financial impact,” our con- 
versation is naturally directed towards diagnosing that 
disconnect. If we’re not even successfully completing our 
“work,” or if the features that we made aren’t being used 
often, those are immediate facts and within the team’s 
control, and metrics should reveal it, regardless of down- 
stream consequences on revenue. You can’t argue against 


solving for the customer! 


4 Blackberry’s revenue continued to grow for two years after the 
iPhone launched; other KPIs were changing far more rapidly, 
and therefore were even more important to track. 


Notice that often the “definition of success” also resides 
in those lagging indicators. This is another traditional 
cause of confusion, often articulated as “we should cele- 
brate outcomes, not work.” It is true that if our work 
doesn’t result in the desired outcomes, we’re not finished 
yet. But, if we believe that the world is inherently unpre- 
dictable, that not all work will yield a large outcome, that 
outcomes require a combination of execution and luck, 
then we should agree that the job of an agile team is to 
continuously tackle that complex challenge, as opposed 
to expecting every idea to consistently produce the de- 


sired outcome. 


Seeing how all the metrics are performing gives the team 
and stakeholders the information needed to completely 
understand what’s going on. Which is the whole point of 
KPIs. 


The agile team must honestly and clearly measure both 
direct results and lagging outcomes. They are account- 
able for all of it, and measuring is required for account- 
ability. But “not yet achieving the outcome” is not a com- 
plete failure, but rather a learning that will shape the 
work that will be executed in the next two weeks. 


If the result confirms the 
hypothesis, then you’ve 
made a measurement. If 
the result is contrary to the 
hypothesis, then you’ve 
made a discovery.” 


—Enrico Fermi 


The axis of “hidden, internal, external, 
and strategic” 


We control what we do in our sprint; we don’t control 
what customers do. The company controls what other 
teams are doing; the product team doesn’t directly con- 


trol that (much to the chagrin of product managers). 
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It’s useful to draw a bright line between what is external 
and what is internal. Often completely different people 
work in these two domains, e.g. product, design, support, 
and sales working directly with customers, whereas engi- 
neering, infrastructure, and security work inside the com- 
pany. The diagram helps us appreciate everyone’s role, 


and use the right metrics for the right things. 


Furthermore, some internal activities are close to the cus- 
tomer (e.g. releasing new features), while others are far 
away (e.g. applying a security patch). If all our work is 
invisible, we have a problem: Customers perceive a stag- 
nant product, competitors appear to be moving faster, 
sales doesn’t have new things to say. On the other hand, 
if we value only the visible things, we end up with a bad 
product, with tech debt and unhappy engineers with 
slow delivery due to an under-invested foundation. The 
diagram makes this clear, honoring all of these important 


types of work. 


The most valuable, strategic outcomes are often even 
more distant from the product team, whether because 
they are down-stream, or because they are second-order 
effects for the customer. We control “satisfaction” more 
than self-motivated external “advocacy,” yet the latter is 
clearly not only the ultimate measure of the success of 
the product, but also drives efficient growth. Product 
teams should take ownership of creating those outcomes, 
while not allowing those lagging, multi-factor metrics to 


be the only way we measure progress. 


The most valuable thing is for the customer to achieve 
their own ultimate goal, as defined by the Needs Stack. If 
the customer’s business doesn’t thrive, theyll stop paying 
for your software, no matter how good the software is. 
While of course the customer’s business is again a multi- 
factor, lagging metric, where nearly all the factors are 
outside of your control, it’s still ultimately the greatest 
form of value. Even if you can’t control it, you can notice 
the attributes of customers who tend to thrive, and direct 
your marketing, sales, and features towards that subset 
of the market, yielding higher growth and retention, and 


likely higher profitability. 


Checklist for great metrics 


From SMART goals to FAST goals to North Star Metrics, 
there’s plenty of prior art on how to pick good metrics. 


This is my own list. 


Defined in normal language 

“Customers are using feature X” makes sense. “Total 
unique IPs which caused at least one event from P Q, 
or R to fire in our analytics system over a rolling 14 
day period, divided by total unique IPs from the same 
system in the same period” is a precise way of measur- 
ing “using feature X,” but it’s too hard for normal peo- 


ple to scan and understand. 


Defined precisely 

In the previous example, you need that technical defi- 
nition also. Frequently that definition is where we re- 
alize either (a) we can’t get exactly the metric we 
wanted, or (b) we have to do engineering work before 
we get the metric we want. This is especially impor- 
tant with concepts like “cancellation rate” or “cost to 
acquire a customer” which can be defined in myriad 
ways; it’s often useful to use different precise defini- 


tions for different metrics within the same company. 


Matches the intent 

Often the technical metric doesn’t actually measure 
what we stated in plain language. For example, we in- 
tend to measure “User Portal usage” but instead we 
measure “User Portal logins,” which only triggers 
when someone’s session expires. It is common to want 
to measure P but P is too difficult or maybe even im- 
possible, so we pick a proxy metric Q. That can be OK, 
but make sure the proxy really does measure the in- 


tended concept. 


Causes action 

If the metric does something differently from what we 
expect, would we act? Would we re-plan the next 
sprint, or even interrupt the current one? If the an- 


swer is “no,” it doesn’t belong on our main metrics 
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board. It might belong on an operational board, if it’s 
explanatory, helping us to understand how things are 


functioning. 


Obvious what “good” look like 

Not all metrics should be attached to explicit goals, 
because some are for monitoring and understanding 
the situation, as opposed to something we're actively 
trying to change or maintain. However, it should be 
obvious “what good looks like.” That could be as sim- 
ple as “not changing,” or directional rather than spe- 
cific (e.g. “usage increasing” as opposed to “usage go- 


ing up 10% month-over-month”). 


Measures “what is happening” (not “work”) 

You already track work; metrics shouldn’t duplicate or 
summarize that. Metrics are about “what is happen- 
ing” around us—the dials on the airplane dashboard, 
not the actions the pilots are taking. (Exception: 
Metrics that explicitly measure whether we are com- 
pleting a volume or quality of work.) If the metric can 
be moved by, or applies to only one possible course of 
action, it’s measuring work and even presupposing so- 


lutions, instead of measuring “what is.” 


Measured easily 
Many metrics are useful, but remain un-updated in 
spreadsheets because it’s too hard to get them. Best is 


automated; second-best is manual-but-trivial. 


Measured frequently 

Daily is best. One of the advantages of “rolling N days” 
is that you can update it daily, yet still think in units 
like “week” or “month” if that’s sensible. Caveat: if the 
number doesn’t naturally change frequently, then it’s 
not important to measure it frequently. Still, in that 
case you won’t check it often, which diminishes its 


value. 


Stable definition and applicability 

Measure things whose definition is stable over time, 
ensuring that any observed changes are the result of 
deliberate actions or environmental shifts, rather than 


random fluctuations or alterations in the nature of 


measurement itself. This stability allows for meaning- 
ful month-to-month comparisons and more accurate 


assessments of strategies and outcomes. 


Uses common definitions when possible 

It’s tempting to invent your own metric, even when 
there are so-called “industry standards” or “best-prac- 
tices.” Standard definitions might not be appropriate 
for your business, or violate one of the other rules 
above. Inventing your own language is also a form of 
team-cohesion. However, reinventing the wheel makes 
it harder for others to understand what you’re doing 
and precludes using benchmarks to see whether your 
metric is “good” objectively. There are even metrics 
where special definitions hide the signal; sometimes 


these are even used for deception”. Don’t do that. 


> Common examples are revenue-recognition, classifying costs in 
or out of COGS/GPM, and conveniently leaving certain costs 
out of EBITDA. 


Signal at least 2x stronger than noise 
Metrics often vary for reasons unrelated to the under- 
lying signal. 


Monthly revenue is like this—new, upgrade, and can- 
cellation alike. Calendar months vary their number of 
days by +5%. Furthermore, daily numbers can vary by 
2x between a week-day and week-end, and calendar 
months vary in the number of weekdays by +10%. So, 
if you’re tracking something like new revenue per 
month, even a real change as large as 10% is the same 
size as the noise, so you can’t actually tell if there was 


a real change. 


Sometimes a clever alteration to the definition can re- 
move noise. For example, “new revenue over rolling 


28 days” eliminates the two factors just mentioned, 
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and is fairly close to a calendar-month worth of rev- 
enue®. Or if noise is infrequent, something like “medi- 


an” or “95%tile” can ignore outliers. 


But often noise is less predictable. In that case, it’s 
useful to ask how much is noise, and whether it’s so 
much that the metric isn’t useful. If you don’t know 


how much is noise, it’s probably a bad metric. 


6 To make it an “average calendar month,” you can multiply your 
trailing 28-day figure by 52/48, which scales it to the average 
number of weeks in a month. 


I hope this system will be as useful to you as it has been 


for us. 


Many thanks to Jaakko Piipponen, Jonathan Drake, and Vinod Valloppillil 
for contributing insights to early drafts. 
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In command 


by Jason Cohen on July 30, 2023 


Being “in control” is impossible, perhaps not even desirable. 
Being “in command” is ideal: honest, introspective, agile, 
aware, and proactive. 


"Stocks rose slightly in early trading, then 
plummeted on news that stocks rose 
slightly in early trading." 


When a founder or CEO or team leader is “in command” 
of her business, I don’t worry about that business, regard- 


less of its challenges. 


Being “in command” doesn’t mean you always meet your 
own expectations for yourself or your team. It means you 
have expectations of yourself and your team, and when 


you don’t meet them, it bothers you, and you react. 


Being “in command” doesn’t mean the metrics are always 
healthy, nor even that they're moving in the right direc- 
tion. It means you’ve already thought about what to mea- 
sure and you’re watching the things that really matter. 
When metrics go awry, you're the first one to identify 
that, and are already taking action in proportion to their 


importance and how far off the mark they are. 


Being “in command” doesn’t mean the product has no 
bugs. It means you have a list of 100-1000 bugs, and the 
reason you're not working on them is that you’re working 
on the top three that matter most. 


Being “in command” doesn’t mean you're always the 
smartest person in the room. It means you've hired a 
team of great people who can challenge you and bring 


diverse perspectives to the table. 


Being “in command” doesn’t mean you never make mis- 
takes. It means you’re proactively on the look-out for 
mistakes, and call them out when you find them, so that 
now you can rally many people in finding solutions, and 


make a change. 


Being “in command” doesn’t mean you never feel over- 
whelmed. It means you recognize when you're over- 
whelmed, take steps to manage your stress, and seek 


support when needed. 


Being “in command” doesn’t mean there are no problems 
with any team member. It means you’ve already talked to 
that person about it, and you’re working with them to 
change. Whether that means changing and staying, or 
changing by leaving. It means you’re proactively and in- 
tentionally solving for fulfillment for the whole team, 


which includes yourself. 


Being “in command” doesn’t mean there are no existen- 
tial threats looming over the business. It means you’ve 
articulated what those are because you sought them out 
and faced them head-on, and you’re mitigating the more 


likely ones with a few, logical actions. 


Being “in command” doesn’t mean customers aren’t leav- 
ing due to lack of features. It means you’ve identified 
what those features are, which ones are consistent with 
your strategy, which ones lead to the ultimate benefits 
the customer is seeking, and that you’re working on one 


or two of them right now. 


Being “in command” doesn’t mean youre always popular. 
It means you’re willing to make tough decisions, even if 
they're not always well-received, because they're in the 
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best interest of the customer, the team, and the business. 


Being “in command” doesn’t mean you’re constantly 
telling people what to do. It means you're acting as an 
editor, not a writer, unless being a writer is necessary to 
make progress. And if that is happening constantly, that 
you make a change on the team so that it’s not happen- 
ing constantly. That might mean changing someone else, 


or changing yourself! 


lig you’re constantly having to do someone else’s work, that’s 
your fault. Either you’ve hired incorrectly, which is your fault, 
or you have hired correctly and you're not allowing them to do 
their job, which is your fault. 


Being “in command” doesn’t mean your strategy is com- 
pletely correct. It means you are adhering to the charac- 
teristics of great strategy. You have one, that is written 
down, that a normal person can understand, that people 
genuinely try to follow. When strategic flaws become ap- 
parent, you write those down too, and work on one or 
two at a time, updating your strategy calmly but 


purposefully. 


Being “in command” doesn’t mean surprises never occur. 
It means when they do occur, it’s not due to negligence. 
It means something actually changed, or an important 
new fact was uncovered. Even better, if you were inten- 
tionally seeking new information, and thus seeking 
surprise. 


Being “in command” doesn’t mean you win every sale. It 
means you seek to understand the patterns in the wins 
and losses, and when those intersect with something con- 
sistent with your strengths and strategy and thus where 
you should win, you make changes so you can win the 
next such deal. 


Being “in command” doesn’t mean every A/B test is a 
success. It means you’re running A/B tests and honestly 


assessing the results, so that you improve over time. 


Being “in command” doesn’t mean you don’t have a long 
backlog, most of which you'll never get to. It means 


you’ve identified the few Rocks that are your bets on how 


to win strategically, and you have a system for prioritiz- 
ing the rest, which also means doing almost none of the 


rest. 


Being “in command” doesn’t mean you know everything 
about your target customer. It means you are actively 
seeking out what they say and do, so you can understand 


them better, so you can make better decisions. 


Well when events change, I change 
my mind. What do you do?” 

—Paul Samuelson, winner of 1970 
Nobel Prize in Economics, about how his 
models of inflation during WWII kept 
changing over time, and he was criticized for 


“not being able to make up his mind.” 


Being “in command” doesn’t mean you never change 
your mind. Indeed, a mind that never changes, is very 
likely wrong. It means you wrote down what you think, 
so that it’s more obvious when your mind needs to 
change, and then you communicate that change—both 


what and why. Even if just to yourself. 


Being “in command” doesn’t mean you do everything 
that stakeholders or even executives ask you to do. It 
means you listened with curiosity and empathy to what 
has been asked, having a system to reject bad requests 
and prioritize good requests, communicating that result 


and the rationale behind it. 


Being “in command” doesn’t mean you’re “in control” of 
everyone and everything. It means you’re proactively 
seeking the truth, 


working rationally, and 


communicating. 


This is how you create autonomy, coupled with account- 


ability, with realistic expectations. 
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Strive to be “in command.” 
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What makes a strategy great 


by Jason Cohen on August 13, 2023 


Most so-called “strategies” are vague, wishful thinking, 
written once and never seen again. Don’t do that. These are 
the characteristics of great strategy. 


— 


“I’m looking for a strategy to leverage our core competencies 
with big data across multiple synergized paradigms. 
Or something that rhymes. Either way.” 


Strategy is: How we will win. 


You can debate the form a strategy should take, whether 
a four-sentence “master plan” or a few dozen bullets or a 
six-pager or an eighty-pager or a template like 
Salesforce’s V2MOM or a Notion template you found on 


the Internet or answering three questions from a Twitter 


pundit or some sort of “Canvas.” Regardless, its job is to 


communicate “how we will win.” 


There are a lot of documents titled “Strategy,” but very 
little great strategy. The formula for great strategy isn’t 


decided by the format of the output document. 


Greatness needs luck, but it’s 
never by accident.” 


—Unknown 


For strategy to correctly determine and communicate 
“how we will win,” it must tackle the reality of complex 


systems, it must parry the follies while exploiting the 


strengths of human nature, and it must justify and spell 


out the decisions which lead to the desired outcome. 


Great strategies accomplish this with the following 


characteristics: 


e Simple: Reshapes complexity to be manageable and 
actionable. 

e Candid: Dares to spotlight the most difficult truths. 

¢ Decisive: Asserts clear decisions and accepts their 
consequences. 

e Leveraged: Magnifies strengths into durable 
competitive advantage. 

e Asymmetric: Defeats uncertainty with higher upside 
than downside. 


¢ Futuristic: Solves for the long-term. 


Without these qualities, the so-called “strategy” is at best 
a plan; at worst, it’s wishful thinking masquerading as 


“vision.” 


Simple 


Simple can be harder than 
complex: You have to work hard 
to get your thinking clean to make 
it simple. But it’s worth it in the 
end because once you get there, 
you can move mountains.” 

—Steve Jobs 


The world is complex, and therefore difficult to reason 
about. Strategy intentionally omits detail in exchange for 


clarity. Armed with a simple narrative, mere mortals can 
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achieve understanding, and make every-day decisions 


that remain aligned to that narrative. 


No one reads anything. You’re in the Top 1% just for 
reading this sentence. No one remembers anything. 
Certainly not an 86-slide PowerPoint! So the strategy has 
to be simple—even simplistic—to have a chance at being 


read or remembered. 


1 Long supporting documents are useful, because they explain 
and justify complex topics, inspiring conviction even in people 
who don’t read it but are impressed by its size. Even the fa- 
mously pithy Tesla “Master Plan” strategy consisting of just 39 
words was preceded by 1200 words of justification. 


People are mired their day to day work; only when the 
strategy is simple, does it have a chance of being incorpo- 
rated. Only when the context is over-simplified, having 
made scrutable the complexity of the real world, can we 
easily explain “why,” and retell those stories in our week- 
ly meetings and prioritization sessions. Those are the 
places where strategy lives and breathes, where teams 
can move quickly, independently, fulfilling their individ- 
ual missions with a minimum of coordination, yet all sup- 


porting a common theory for how we will win, together. 


The bigger the project, the more 
we need a reminder of the goal, 
and simple things to execute 
today.” 

—Warren Buffett, shareholder letter, 
1982 


If the strategy is simple, it might appear obvious. If it is 
obvious, it might appear uninsightful. Humans assume 
that complex puzzles require complex answers, when in 
fact often the simplest answer is the best answer” “Obvi- 
ousness” is a sign of a strategy that not only is easy to 


communicate and execute, but also believed. 


2 See, for example, the Midwit Fallacy, or how small sets of equa- 
tions explain a wide range of physical phenomena. 


Its job isn’t to be non-obvious, but rather to cleanly speci- 
fy what is most important. It might be obvious to do X, 
but Y and Z are also “obvious,” so by selecting X and not 
Y nor Z, you have created focus, and specified “how to 
win.” 

Simplicity at its worst becomes reductive—overlooking 
complexity rather than tackling it, resulting in conclu- 
sions that, while admittedly simple and clear, are wrong. 
We cannot pretend that complex problems can always be 
waved away by simple statements. The process of strate- 
gy creation must indeed tackle the complexity of the 
world, but the output of that process is a document that 
frees everyone else from having to re-solve every puzzle. 
Like a street map, we omit detail in exchange for clarity 
of the most important context and routes. Simplicity that 
ignores reality is reductive, but simplicity that arises from 
an exceptional summarization of having already pro- 


cessed the messy, complex world, is elegant. 


Characterizing the world in a few, simple, clear asser- 
tions, and solving our challenges with a few, clear direc- 
tives of what we must do, is required for a strategy to be 


“how we will win.” 


Example: The most famous sports coaches seem to credit 
their success to a few simple principles. They must be simple 
for a team to remember them, especially when they’re tired 
and battered near the end of a game, as in this description 
of the mentor of one of America’s most famous football 


coaches: 


“Blaik’s signature talent was using all this data to cre- 
ate something clean and simple. He had what 
Lombardi called “the great knack” for knowing what 
offensive plan to use against what defense, and the 
“discarding the immaterial and going with the 
strength.” All the detailed preparations resulted not in 
a mass of confusing statistics and plans, but in the op- 
posite, paring away the extraneous, reducing and refin- 
ing until all that was left was what was needed for that 
game against that team. It’s a lesson Lombardi never 
forgot. 

—David Maraniss, When Pride Still Mattered: A Life of 
Vince Lombardi, 1999 
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Candid 


See, and don’t be afraid to see 
what you see.” 
—Ronald Reagan 


Strategy must lay bare the most frightening, embarrass- 
ing realities. It must face the truth that we all avoid fac- 


ing during the struggle of daily work. 


If the strategy doesn’t expose—and then solve—the most 


brutal facts, the strategy is wrong. 


In particular, the strategy must diagnose the primary 
challenges facing the business, even if the facts are so 
scary that it seems hopeless. Existential threats are the 
most important to articulate. 


Too often a strategy claims “threats” that are lazy, gener- 
ic, non-actionable pseudo-concerns that nearly every 
company could claim, like “Google could copy us” or “A 
new startup could invent a great product and get huge 
funding” or “A massive security breach could cause half 
our customers to leave.” Real threats are either happen- 


ing now or are at least 70% likely to happen. Real threats 
are specific, ideally backed by data that proves they are 
happening, e.g. your market is shrinking; a competitor 
has accelerating market share; cancellation rates prove 
that even paying customers don’t value the product. Real 
threats are written in the present tense because they are 


happening, not hypothetically with “could” or “might.” 


When the puzzles are clear, everyone can help solve 
them. When the puzzles remain hidden, what’s the 


chance that they'll be solved by accident? 


Average players want to be left 
alone. Good players want to be 
coached. Great players want the 
coach to tell them the truth.” 
—Nick Saban (won more US national 


college football titles than any other coach) 


If you’re worried someone might leave the company 
when they hear how scary it is, maybe they should leave 
the company. These are the challenges we’re facing to- 
gether; if they’re not willing to solve them, they need to 
make space for someone who will relish the challenge. 
Not just for the company’s sake but for their own happi- 


ness and fulfillment. 


“Let's say honesty isn’t going to work. What's the second 
best policy?” 


Facing the truth, being specific about current reality and 
about what needs to be done, is required for a strategy to 
be “how we will win.” 


Example: In a story retold here, originally from Jim 
Collins’s book Good to Great, A&P and Kroger were success- 
ful grocery stores in the 1960s who both possessed the data 
showing that their business models were becoming obsolete. 
They each discovered the new, correct business model 


through real-world experimentation, but only Kroger was 
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willing to face that truth and change their strategy. Despite 
being half the size of A&P Kroger increased its value 100- 
fold, while A&P shrank, then went bankrupt. 


Decisive 


A strategy asserts a set of justified, self-consistent deci- 
sions, such as: 


e Which subsets of the market to target 
...and conversely which we'll ignore, even if some of 
those sign up as customers anyway, and ask for things 
we’re not going to do, and then cancel in anger 

e Which customer personas are most important to 
delight 
...and conversely which will dislike our product, 
causing us no dismay, whose feature-requests we will 
quickly close as “won’t do” rather than wring our 
hands at all the features we still need to build 

¢« How to position against the competition 
...and conversely where the competition will be 
stronger, unlike those fake-news marketing charts 
comparing our product with the competition, where 
only our product scores 100% along every dimension 

e« What we value (e.g. quality, service, speed, design, 
compatibility, lock-in) 
...and conversely what we will give up, e.g. releasing 
features faster but of lower quality, or being infinitely 
extensible versus top-to-bottom thoughtful design 

e« What we must (and must not) build, to pay off that 


positioning and win those customers in that market. 


You can do anything, but not 
everything.” 
—David Allen 


There are both positive and negative second-order conse- 
quences of any complex decision. If these aren’t identi- 
fied—in particular, if the problematic consequences 


aren’t embraced within the strategy—then it’s not a clear 


decision. When those consequences inevitably arise, the 
team must be able to say “we expected that” rather than 
“we have to address that” or even “this is a signal that 


the strategy is wrong.” 


One decision in every strategy is what market and cus- 
tomer-segment the company will target. Customers out- 
side that target segment will inevitably buy anyway. Then 
they'll complain about missing features, awkward UX 
workflows, missing integrations, high prices, and more. 
Then you'll log those complaints into the bug-tracker and 
product-management Miro boards and start prioritizing, 
whereas in fact these ideas should be ignored so that the 
team can focus on winning the target customers. This is 
difficult to remember when the non-target customers 
complain on Twitter, supply low NPS scores, drag down 
your average star-rating on review sites, and cancel at 
high rates. Only with a strategy that has clearly articulat- 
ed not only the decisions, but also the negative conse- 
quences of those decisions, can everyone stay true to 
those decisions even when emotions are running high 


and paying customers are canceling. 


A common tactic for avoiding making a decision is to use 
non-specific language. “We will leverage synergies to cre- 
ate unique solutions” is, in fact, a good thing to do, but it 
doesn’t specify which synergies to leverage, what_is 
unigue about it, or what the unique solution is. Fluffy 
language is a hallmark of indecisiveness, and therefore of 


bad strategy. 


Strategy is a set of interrelated 
and powerful choices that 
positions the organization to 


win. 
—Roger Martin 


Decisions and consequences must at minimum be self- 
consistent. If you’ve decided to have a low price, you 


can’t also have white-glove service. If you’ve decided that 
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everything requires high-quality design, you can’t also re- 
lease features faster than the competition. Or, perhaps 
you can creatively build a solution that does say “yes to 
both” of those things, but only by also accepting addi- 


tional constraints that resolve the conflict. 


Better than “self-consistent” is “mutually-reinforcing.” 
This means that one decision makes another more pow- 
erful, or easier, or less expensive, and vice versa, so that 
adhering to both makes you far stronger than having 
only one. For example, deciding to have only a few fea- 
tures, and also amazing design. Normally customers 
might not put up with less functionality, but if the design 
experience is exceptional, they might be happy with 
something that “does only a few things, but so delightful- 
ly!” And vice versa: It’s easier to execute on great design 
when you don’t have to tackle a complex product with 
tons of use-cases and personas and functionality. At the 
end of this article, you’ll find a complete, powerful exam- 
ple of mutually-reinforcing decisions creating a durable 


(60-years!) competitive advantage. 


The difference between successful 
people and really successful people 
is that really successful people say 
no to almost everything.” 

—Warren Buffett 


You know the decisions are strong when they—and their 
consequences—cause you to say “no” to most things, in- 
cluding things which otherwise sound reasonable. An ex- 
ample of a strategy that makes a decision but also ac- 
cepts negative consequences can be found at the end of 
this article about Moats; an example of how to make 
sales pitches while accepting negative consequences can 
be found in this article; several examples of how YouTube 
used a single, strong decision with multiple major down- 


stream consequences are detailed in this article. 


“I haven't yet decided if I’m staunchly in favor, 
or staunchly opposed. But rest assured, 
however I vote, it will be staunchly.” 


When a strategy has articulated clear decisions, including 
the major consequences, especially accepting the nega- 
tive consequences, causing us to say “no” to pretty-good 
ideas in order to make room for the very-best ideas, and 
to ignore the wishes of paying customers if they’re not 
the target paying customers, it is truly prescribing “how 


we will win.” 


Example: Craig’s List is classified ads on the Internet. Their 
strategy includes crisp decisions like valuing consistent user 
experience more than great design (the website looks like 
it’s from 1995, because it is), and user-accessibility over 
monetization (“Craigslist president Jim Buckmaster has 
stated that creating a superior user experience is more im- 
portant to the company than making money [source]”). 
The strategy is clear and differentiated, which should be 
seen as a strength, but it has bothered pundits for twenty- 
five years, who have therefore predicted its demise: “The 
main obstacle to sustainability and growth at Craigslist is 
likely the company’s and its founder’s strong principles 
valuing customer-offering over monetization, trusting con- 
sistency over innovation. Although admirable in many 
ways, the issue is that without innovation, the company’s 
customer-offering will soon no longer be strong enough to 
stay relevant, undermining its very  well-intended 
purpose."—HBS. But crisp decisions make for a great strat- 
egy, even if controversial. Despite that well-reasoned death- 


sentence from HBS, despite others pointing out that dozens 
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of successful companies have made sections of Craig’s List 


theoretically obsolete, despite data that predicts Craig’s List 
“demise”, despite protestations that Craig’s List could be a 
“gold mine of revenue, if only it would abandon its commu- 
nist manifesto,” (i.e. if only it would abandon its strategy), 
it continues to be one of the most-visited sites in America 
with revenues of $1,000,000,000 in 2020 with a staff of 
just fifty people, even after twenty-five years of Internet 


evolution. 


Leveraged 


Be yourself. Everyone else is 
taken.” 
—Oscar Wilde 


If Sandra is a great singer, and Peter is a great pianist, 
then a good idea for a duet would be for Sandra to sing 
and Peter to play piano. A bad idea would be the other 
way around. Great strategy leverages the strengths and 
assets of the organization; a bad strategy asks the organi- 
zation to win even while acting unnaturally, often under 
the cover of “overcoming weaknesses.” 


“Leverage” means generating a large effect from a rela- 
tively small effort, where time and dollars are far more 
effective than one might expect, because we are riding 
tailwinds of natural abilities or hard-won assets, rather 
than fighting a battle against so-called “weaknesses.” You 
know you’re leveraging strengths when you see other 
people shake their heads in amazement at how much you 


accomplished in so little time. 


It’s good to leverage strengths. Much literature on strate- 
gy dwells on how to create moats—permanent competi- 
tive advantages—but so many organizations still aren’t 
leveraging the _ straightforward, undifferentiated 
strengths that they possess. They expend most of their 


energy shoring up “weaknesses,” which despite their ef- 


forts will at best become “less weak,” but never become a 
strength. Whereas applying that same energy in leverag- 
ing their strengths will have a large positive effect. It’s 
even just more fun to play to your strengths instead of 


wallowing in weaknesses. 


It is of course better when the strength is differentiated 
from the competition. This is especially obvious in a ma- 
ture market where everyone is saying the same things on 
their home page, pricing the same way, and different 
only in tertiary characteristics. Winning in your own way 


can defeat “better” competitors. 


It’s even better when that differentiation is durable over 
time. It has never been more difficult to establish a per- 
manent advantage, when all software can be reproduced, 
all business models can be replicated, and the entire 
world is both your market and your competitors, which 
makes it all the more important to decide what one or 
two moats you will build. The strategy is the place to 


name those moats. 


This companion article on “leverage” provides expand- 
ed examples and ideas for all of the above. 


Tailoring the decisions for the strengths of this organiza- 
tion, avoiding (rather than reversing) weaknesses, even 
better when the strength is differentiated, identifying and 
investing in durable differentiation, so that moats are 
constructed in the long run, is required for a strategy to 
be “how we will win.” 


Example: Zappos started in 1999—early Internet, no social 
media, few people with broadband, few people comfortable 
buying anything online, much less buying shoes, which for 
100 years was bought only after first trying them on in per- 
son. Zappos started with a differentiated insight (the com- 
panion article explains how “insights” can be leverage): 
That people would actually buy shoes online, under the 
right circumstances. They built a moat out of a combina- 
tion of surprising decisions (also explained in the compan- 
ion article): (a) postage-paid returns even 364 days after 
purchase, (b) legendary support who did things like buying 


pizzas for customers, (c) a corporate culture of employee 
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empowerment; it takes that combination of strong, expen- 
sive decisions to create the conditions by which people 
would dare to buy online, thus unlocking an entirely new 
retail channel. Only by having all three of those compo- 
nents did the moat work; if you don’t empower employees, 
they can’t order pizza on the spur of the moment; if your 
return policy isn’t ridiculously generous, people will be too 
scared to order shoes sight-unseen. Because their vision and 
decisions were so incredible (literally “not credible”), com- 
petitors laughed at them while they enjoyed a constant 
stream of positive press. The result was the most successful 
online shoe store for a decade, reaching annual revenues of 
$1B before its tenth birthday. 


Asymmetric 


: WIND 


A sailboat always moves forward. 


This remarkable fact is due to its asymmetrical shape: it 
is pointy at the front and flat in the back. This creates re- 
sistance to moving backwards, but a natural ease in mov- 
ing forward. Even when the water is randomly undulat- 
ing, “backward” forces are muted, while “forward” forces 
are allowed, so the boat glides forward. Asymmetries can 
amplify positive effects while muting negative ones, re- 
sulting in a net-positive force even under conditions of 


random inputs. 


Great strategies prescribe activities that always move the 
company forward, despite the inevitable bad luck and 


setbacks. To do that, the activities must exhibit asymme- 


try, where the upside vastly exceeds the cost, so that even 


if you took 2x longer to achieve 50% of what you expect- 
ed, you still win. 


If riskier investments could be 

counted on to produce higher 

returns, they wouldn’t be riskier.” 
—Howard Marks 


This is the mechanism behind Venture Capital portfolios, 
which are investments in a slate of early-stage startups. 
The worst-case outcome for each bet is that they lose 
100% of their investment, but in the best case they can 
gain 10,000%. A few large successes more than make up 
for the many failures, so the portfolio in total comes out 
positive. Investors call these “asymmetric _ bets.” 
Economists call this “convexity.” 


Strategy must create a portfolio of bets having this “VC- 
like” asymmetric quality, whether for a small startup try- 
ing to find product/market fit or a mature company en- 
tering new markets. A sign of a bad strategy is when suc- 
cess requires everything to go right. With a set of asym- 
metric bets, the successes render the failures moot, and 
so the unpredictable waves crashing into the boat still re- 


sult in forward motion. 


This companion article details strategies that work 
well in an unpredictable world. One of the few 
strengths of a new startup is that it can learn faster, react 
faster, deliver faster; startups are favored when the wa- 


ters are uncertain. 


One form of asymmetric bet is entering a large and grow- 
ing market. Besides the obvious benefits? there is the 
asymmetry of optionality: There are many niches to ex- 
ploit, many possible ways for a product to deliver value, 
many marketing and sales channels, and there’s more of 
all of it every year. Because there are many options to try, 
there are many ways to succeed; if your first few ideas 


don’t work, the next one might. Waves on the boat. 
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3 Potential customers are already spending money, which means 
budgets are pre-allocated and pricing structures are well-un- 
derstood, the press is already talking about it, marketing chan- 
nels already exist, and the “pie” is growing, so even a small 
slice of the pie automatically grows. 


The most powerful force in the 
universe is compound interest.” 
—Albert Einstein 


Another kind of asymmetry is a process that compounds, 
meaning that the more of it there is, the faster it grows. 
Things with this characteristic naturally grow larger than 
anything that grows in a more linear fashion, even if they 
start out small. Examples are customer retention, cus- 
tomer upgrades, and employee retention. Another exam- 
ple are growth-vectors that are proportional to the size of 
the current customer base, such as word of mouth refer- 
rals and viral products (e.g. once you join a social plat- 
form or collaborative online tool, you tend to get other 
people you know to join as well). Most things that grow 
non-linearly don’t_grow exponentially—that’s normal, 
and still a great strategy. A common example is a reseller 
channel, because each reseller could bring you a number 
of clients over time, so when the number of resellers 
grows by Nn, the number of potential customers they 
bring grows by N x M. For example, Inuit grew 
Quickbooks by providing software to CPAs that made 
their life easy, but only if their clients used Quickbooks. 
The CPAs started requiring their clients to buy 
Quickbooks, and so a smaller number of CPAs created a 


much larger number of Quickbooks sales. 


In the negative, companies also face asymmetric threats. 
With ten competitors, only a few need to become break- 
out winners to pose a significant challenge. Like the fa- 
mous statement by the IRA after a bombing failed to kill 
British prime minister Margaret Thatcher: “Today we 


were unlucky, but remember we only have to be lucky 


once. You will have to be lucky always.” A strategy that 
places a number of asymmetric bets doesn’t need to be 


lucky always. 


Every plan will face challenges, both foreseeable and bad 
luck. If everything has to go right for the plan to succeed, 
it won’t succeed. Whether a single investment has asym- 
metric upside, or a portfolio of bets collectively has large 
upside, exploiting asymmetries maximizes the chance 
that the strategy will succeed despite the inevitable tra- 
vails and uncontrollable luck, and thus is vital to “how 


we will win.” 


Example: Amazon is the cliché example, but only because 
it’s apropos, even ignoring AWS. Originally a book-seller 
only, they have never stopped taking bets, whether on adja- 
cent markets (e.g. selling electronics), reselling internal sys- 
tems (e.g. Amazon warehouses, logistics, robotics, fulfill- 
ment, were all broken apart and sold as separate products), 
or entirely new kinds of product (smart phones, cloud com- 
puting). More impressive is when the new bets purposefully 
disrupted previous businesses; according to early employee 
Andy Johns (article, podcast), Bezos was so adamant that 
the Kindle be successful, despite the fact that it would hurt 
their physical book business, that he assigned the Kindle 
project to the executive who was currently over the book 
business, saying “Effective tomorrow, your job is to kill your 
old business with a Kindle.” The key thing is: many of 
these bets failed, and failure was expected. Failure isn’t 
desirable, but each bet had out-sized potential upside and 
budget-able downside. Even Kindle was unprofitable for 
years and the Amazon Fire Phone was an abject failure. 
Bezos explains in his own words that their strategy is exact- 
ly the theory of a portfolio of asymmetric bets with expected 


failures: 
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“If you’re going to take bold bets, they're going to be 
experiments. And if they're experiments you don’t 
know ahead of time if theyre going to work. 
Experiments are by their very nature prone to failure. 
But a few big successes compensate for dozens and 
dozens of things that didn’t work.” 


‘Tve made billions of dollars of failures at 
Amazon.com. Literally. You have to be super clear 
about what kind of company you're trying to build. ... 
We said we were going to take big bets. We said we 
were going to fail.” 


—The Guardian, 2014 


Futuristic 


The best time to plant a tree is 
one hundred years ago. The 
second-best time to plant a tree is 
today.” 

—A proverb claimed by many peoples, 


across thousands of years 


“Being agile*” 


is a great way to climb the proverbial 
mountain-shrouded-in-fog. Some paths are the right 
ones, but backtracking is inevitable; it’s a sign of puz- 
zling-out, not a sign of failure. The job of strategy is to 
identify which mountain we're trying to climb in the first 
place—the puzzle we’re solving for, the opportunity we’re 
exploiting. If an “agile, self-managed” team climbs the 


wrong mountain, it was all a waste. 


4 Meaning: Don’t pretend you can predict the future, assume the 
quantity of “things we don’t know” is larger than the quantity 
“things we do know,” iterate quickly on hypotheses that you 
proactively attempt to disprove, and adjust in the presence of 
new information. The team that learns the fastest, wins. The 
team that spends three months trying predict the future, is now 
three months behind, and the future still won’t unfold as they 
predicted. 


Strategy looks further into the future than anything else 
at the company. Therefore, it has the responsibility to 
take the long view. Which is especially difficult, as the fu- 
ture is unpredictable, and data tells you about the past, 


but rarely about the future. 


If you can solve a problem in a month, you probably 
should, but that also means it’s not a strategic problem. 
Anything that can be built in three months, isn’t the way 
you will have constructed a moat that will take competi- 
tors years to overcome. Anything that specifies features 
or timelines is a roadmap, not a strategy; a strategy spec- 
ifies market and business outcomes and the primary deci- 
sions and secondary consequences. Anything that speci- 
fies teams or roles or hiring or processes is an operational 
plan, not a strategy; a strategy details the outcomes and 
activities that require the entire company to accomplish 


together, not what one team needs to accomplish alone. 


People naturally get distracted by the immediate, the ur- 
gent, the tactical, the “low-hanging fruit.” Strategy is the 
place where we select the high-hanging fruit, the “impor- 
tant, but not urgent” quadrant of the Eisenhower Matrix. 
Then executed as the Rocks in your “Rocks, Pebbles, 


Sand” prioritization framework. 


Part of explaining “how to win” is defining what the 
“winning” state looks like. Often called the “vision state- 
ment,” it describes an what the world will look like when 


we're successful: 
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Oxfam A just world without poverty 

Habitat for A world where everyone has a decent place to 

Humanity live 

Stripe Increase the GDP of the Internet 

Microsoft A computer on every desk and in every home 

(years ago) 

Tesla Create the most compelling car company of 

(original) the 21st century by driving the world’s transi- 
tion to electric vehicles 

WP Engine Power the freedom to create online 

YouTube Give everyone a voice and show them the 
world 

GoDaddy Radically shift the global economy toward in- 


dependent entrepreneurial ventures 


NoBopy'sS 


"You're right. It's not much of a 
mission statement." 


The Vision statement is often the first sentence the strate- 
gy document, but was the last thing crafted by the au- 
thors of that document. Only once you fully understand 
the challenges you face, the main, coherent courses of ac- 
tions to undertake, and the results you want, can you 


summarize a clear vision of what the future will be. 


Strategy is where we specify the most critical long-term 
challenges facing the company, and the rocks that are the 


most important things, not to win the battles today, but 


to determine how we will have won the war three years 
from now, how we will achieve our vision, how we will 


win. 


Example: After Google purchased YouTube, they set a far-fu- 
ture goal of attaining a billion daily views. This filtered 
down to everything from demands on technical architec- 
ture, to strategic debates such as whether that enormous 
quantity of views could be achieved by user-generated con- 
tent alone or whether they needed to license libraries of ex- 
isting TV shows and movies. At one point they realized that 
the global Internet network infrastructure did not have 
enough capacity to stream a billion views per day, so 
Google invested billions of dollars in fiber and data centers 
to prevent that from becoming a bottleneck—billion of dol- 
lars they would not have spent without a vision of the fu- 


ture that mandated that investment. 


Bad Strategy 


Tell-tale signs of a strategy that lacks these qualities: 


Not simple 

Pages of detail? Slides with more than 20 words. A 
litany of numbers without a narrative explaining what 
insights they create. No diagrams “painting the 
picture,” or diagrams with 20 boxes. Too many points 
for someone to recall from memory. Important con- 
cepts that aren’t summarized by a short phrase that 


people can use as a daily short-hand. 


5 Detail is great, if it is attached as optional reading, expanding 
on a summary. 


Not candid 

Nothing where the future of the company hangs in the 
balance. Nothing that makes the reader say, “Oh wow, 
dang, what are we going to do about that!?” Nothing 
scary that demands action. No serious consequence if 
the directives aren’t followed. A reader who finishes 
the document and thinks, “We're still ignoring the ele- 


phant in the room.” 
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Not decisive 

No clear decisions that would cause us to “easily say 
’no” to many otherwise excellent, reasonable ideas. 
Not obvious what we’re not doing. Non-specific target 
market (e.g. “for everyone”), target customer (e.g. 
“any [title]”), target jobs-to-be-done. Directives and 
headings using the word “and” to expand scope rather 
than limiting it. No negative-but-accepted conse- 
quences of the decisions. Decisions that conflict with 


each other. Decisions that don’t reinforce each other. 


Not leveraged 

Strategy would apply equally well to a competitor, or 
even to a business in another industry. Strategy 
doesn’t call out the special strengths and durable as- 
sets of the organization, or doesn’t explain how to ap- 
ply them to win, especially how it will position against 
the competition. No obvious moats being constructed. 
No network of interlocking decisions that together 
makes the company special. Demanding that the orga- 
nization to overcome more than one or two major 


deficiencies. 


Not asymmetric 

Potential upsides aren’t at least 10x larger than costs. 
Linear cost/reward, or risk/reward. Not creating op- 
tionality in how each aspect can go right, leading to 
one thing that absolutely must work. A course of ac- 
tion that requires multiple different, difficult things to 
simultaneously go right, otherwise the whole strategy 


fails. 


Not futuristic 

Doesn’t describe a specific future destination of the 
company or product. A “vision” that describes what 
the company already does and already is, rather than 
how things will be different once we successfully exe- 
cute our strategy. Doesn’t specify which moats are be- 
ing created, and how. Specifies teams, products, time- 
lines, or features. Deals with temporary challenges 


that can be solved in a quarter rather than long-term 


challenges that will take years to fully overcome. 
Describes how to win this year instead of in three 


years. Relies primarily on data to predict the future. 


Not strategy (bonus) 

Generic statements that would apply equally to nearly 
any company, even in a different field (e.g. grow 
faster, lower attrition, hire the best talent, delight cus- 
tomers, beat the competition). Aspirations about what 
we wish would happen, without specifying how it will 
happen. Financial goals rather than how to win com- 
petitive markets. Plans that are in someone’s head in- 
stead of written down and shared. Written documents 
that aren’t referenced when creating plans. 


A sailboat always moves forward, but if you don’t decide 
on a specific destination, it will end up somewhere, but 


probably not where you wanted to be. 


Time to decide how you will win. 


Postscript: How do I construct a strategy? 


Stay tuned for future articles that lay out a process for 
constructing a great strategy. Partial answers appear in 
The roadmap for Product/Market Fit and Excuse me, is 
there a problem?. 


But don’t wait. A mediocre strategy is still better than no 


strategy. 


Further reading 


e Article: WTF is Strategy? by Vince Law: Defining the 
nested concepts of vision, mission, strategy, roadmap, 
and execution, thereby contextualizing the role of 
“strategy.” 

¢ Book: Good Strategy, Bad Strategy by Richard Rumelt: 
Doesn’t explain how to construct a strategy, but 
terrific observations about what good and bad 


strategy looks like. (Summary by Jeff Zych) 
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¢ Book: Blue Ocean Strategy by W. Chan Kim & Renée 


Mauborgne: Explains how to construct a strategy that 
is not only “different from the competition” (i.e. 
competes better in so-called “red-ocean” force 
competitive markets) but competes in a wholly 
different way, creating a new value-proposition, often 
at higher profit, in so-called “blue oceans” where 
you've defined yourself such that there isn’t any 
direct competition. 

Article: Gibson Biddle’s multi-part series on how to 
build a product strategy that converts “how to win” 
into “what to do” 

Article: Taylor Pearson on optionality, and how to 
apply the Nassim Taleb quote in practice, even 
personally. 

Article: Excuse me, is there a problem? on selecting 
the right “problem to solve” and “for whom” to 
increase the probability of success. 


e Article: Using the Needs Stack for competitive 
strategy on thinking about customer value, markets, 
competitors, and disruption. 

e Article: When customers are “willing” to pay on how 
to create strategies that capture and keep customers 
as your allies, who help you grow and find new 
customers, rather than forcing them to stay through 
coercion. 

e Journal article: The Design School: Reconsideration 
of the Basic Premises of Strategic Management by 
Henry Mintzberg, Strategic Management Journal 
(1990), with a summary of traditional strategy 
construction, a critique of its drawbacks, and 


conditions when it is appropriate. 
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Worse, but unique 


by Jason Cohen on August 27, 2023 


An objectively “worse” strategy can win, if it leverages 
something unique or unexpected. Startups can use this 
concept to beat incumbents. 


“I’m not much for video games, but I like the way 
he moves.” 


A perfect storm created a renaissance in chess: COVID 
driving people to occupy their minds in physical isola- 
tion, the hit Netflix show “The Queen’s Gambit,” and a 
surge of chess Twitch streamers and YouTubers fueling 
that nascent interest in chess with equal parts education 


and entertainment. 


And so beginners learned openings like the titular 
Queen’s Gambit—one of the most popular ways for white 
to open the game. It’s easy to find courses, books, and 
online content about the Queen’s Gambit because it’s 
been popular for more than a hundred years. The top 
chess players in the world today still play the Queen’s 
Gambit; if they play it, it must be good! And it is. 


Except, it’s smarter for beginners to play worse openings. 


“Worse” is an objective evaluation. There are openings, 
like the King’s Gambit for example, which were popular 
in the 1800s but have since been “refuted,” meaning that 
for each move by white, black always has a counter-move 
which results in black ultimately achieving a superior po- 


sition. “Superior” as consistently demonstrated in games 


between top-level humans, as well as computers that are 
orders of magnitude stronger than those humans. Thus, 
the King’s Gambit is objectively worse than the Queen’s 
Gambit. 


So why should beginners use the refuted King’s Gambit 
opening? 


First of all, beginners aren’t playing top chess players; 
they are playing someone near their own strength. At 
their level, the King’s Gambit isn’t refuted, because nei- 
ther player has memorized the sprawling branching se- 
quences of all possible moves and counter-moves and 
counter-counter-moves, ad nauseam. Nor can they figure 


out the best move in the moment. 


Second, and more importantly, exactly because the 
Queen’s Gambit is popular, it’s likely that your opponent 
has also studied it. So it’s quite possible that your oppo- 
nent knows a defense to the Queen’s Gambit better than 
you know the attack of the Queen’s Gambit. As you 
progress in chess, this becomes more and more likely. So 
now the question becomes: Is your preparation—memo- 
rization of the branching forest of moves—better than 


your opponent’s preparation? 


That’s not the sort of “luck” you want to depend on. 


That’s not a good strategy. 


The solution is to learn a refuted-but-not-completely- 
atrocious opening like the King’s Gambit. It isn’t refuted 
at your level, and also there’s almost no chance your op- 
ponent has memorized any specific defense for it, be- 
cause “no one plays that.” So they’re reacting to it for the 
first time, surely not finding all the best counter-moves in 
the moment. But you have memorized a few of the best 
moves. So you have a clear advantage, even though 


youre a beginner. 
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This is why leverage, as defined here, results in a superi- 


or strategy for products and companies, even if many 
things about you, your team, your product, and your 
company, are “worse” than the competition. Playing on 
your own terms changes the game. “Worse, but unique,” 


is in fact “better.” 


A startup is a “beginner” in the market, facing a variety 
of “opponents” who have varying degrees of preparation 
and skill (age, capabilities, customers, brand, funding). A 
startup needs to fight the competitive battle on its own 


terms, not attack where the opponent is already strong. 


Even on a personal level, you must use your leverage to 
do something special, be something special, at the inter- 


needs. 


Be yourself. Everyone else is 
taken.” 
—Oscar Wilde 


Many potential customers won’t want to play your game, 
and you'll feel bad when you lose them to a competitor, 
and when they say mean things about you and your 
product and your decisions. Especially when it’s in pub- 
lic, on review sites and social media. But don’t worry, you 
weren't going to win them anyway—they’re just not into 


you. 


The customers who do want to play your game, will see 
that you’re the best in that game. Even if you’re two peo- 
ple and there’s a dozen competitors ranging from well- 
funded, well-covered-by-the-media startups to old-but-re- 
liable incumbents. If you pick a different game, you’re 
the best at that game, and you can become extremely 


successful | 


lin my case, WP Engine powers as many substantial websites as 
Google Cloud or Digital Ocean, and we still have only 2.3% of 
that market. That means 98% of potential customers are in fact 
not customers, and yet WP Engine is a unicorn. By the way, this 
is another reason why it’s wise to seek large markets. 


Even better than “different,” is to be extreme in that dif- 
ference. Not just a minimal UI, but so minimal it works 
on the command-line. Not just great design, but so re- 
markable people buy it only for that, and it’s written up 
on designer’s blogs. Not just a new algorithm that solves 
an old problem, but one that uncovers new things that no 
one else does, even at the expense of missing things that 
others catch. You can’t do this for all aspects of your 
product and business—indeed, even a single one is al- 
ready powerful—but extremity is how you maximize the 


power of the few things that make you special. 


All those competitors cannot play the game as well as 
you, because you picked something that was not only dif- 
ferent, but extreme. They can’t make every choice you 
made, and since they didn’t, theyre definitionally not the 
best at your game. 


Play your King’s Gambit. Don’t worry if others claim your 
strategy is refuted. It’s not refuted for you. 


Indeed, the fact that they think it’s refuted, is why you 


will win. 
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Stop saying “fail” 
by Jason Cohen on September 10, 2023 


Language shapes our perception of setbacks. Use words 
other than “failure” to describe situations and to suggest the 
next step. 


“Instead of calling it a failure, we just rebrand it as a 
dark and gritty reboot of previous profits.” 


It’s not fail fast. It’s not fail forward. It’s not “fail” at all. 


“Fail” is almost never the right word for what is happen- 
ing. It’s so final, so negative, and it’s a dead end: What 


do you do after a failure? Mope? 


Words have meaning. They can explain what’s really go- 
ing on, and suggest what should be done next. The word 


“fail” accomplishes neither. 


Exploring new territory 

If you’re solving a maze, and a path doesn’t work out, 
you haven’t “failed,” you’ve only backtracked. This is a 
necessary and natural aspect of exploration and creating 
businesses and products; in fact it’s even called the idea 
maze. It often comes in the form of an experiment. The 
result of an experiment is evidence supporting or reject- 
ing the hypothesis. Neither result is a “failure.” A/B test- 
ing is a good practice, even though most of the time you 


will not have stumbled upon an improvement. 


Words you could use instead of “fail” include: experiment, 


test, bet, learning, investigate, trial, retry, probe. 


There are two possible 
outcomes: if the result 
confirms the hypothesis, 
then you’ve made a 
measurement. If the result 
is contrary to the 
hypothesis, then you’ve 
made a discovery” 


—Enrico Fermi 


Revolutionizing 

If you’re creating something new, never accomplished by 
anyone before, and it doesn’t immediately revolutionize 
the industry, you haven’t “failed,” you’ve just not yet 
found the breakthrough. Often innovations require many 
trials before the right solution is uncovered. If the right 


answer were obvious, it wouldn’t be an innovation. 


Words you could use instead of “fail” include: backtrack, 


iterate, trial, attempt, step, miss. 


I have not failed. I’ve just 
found 10,000 ways that 
won’t work.” 

—Thomas A. Edison 


Learning 

If you’re learning a new skill, and you stumble, get con- 
fused, get lost, or think you’ve mastered it only to find 
there’s still a long road ahead, you haven’t “failed,” 


you're growing. No good teacher would get mad at a stu- 
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dent for not learning a skill instantly, or even sliding 
backward. The learning curve is well-understood, and it 


is not a straight line. 


Words you could use instead of “fail” include: practice, 
cultivate, acquire, develop, learning curve, learning mo- 


ment, trial, progression, evolve. 


Iam always doing that 
which I can not do, in 
order that I may learn how 
to do it.” 


—Vincent van Gogh 


Iterating 

Are you iterating on something that is fundamentally 
working, just making it better? This is optimization. Not 
all attempts at improvement will work, or have as much 
effect as hoped, but surely we shouldn’t call ourselves 


“failures” when things are basically fine as they are. 


Words you could use instead of “fail” include: refine, cali- 


brate, optimize, polish, hone, correct, fine-tune, dial-in. 


The only real mistake is the 
one from which we learn 
nothing.” 

—Henry Ford 


Solving problems 

Are you solving a problem, but a minor one, particular to 
your current circumstance, where your high-level strate- 
gy and plan is still correct? Problem-solving in a complex 
space—whether a one-person startup or a thousand-per- 
son organization—cannot possibly proceed with innova- 
tion and speed and yet also no setbacks, no complica- 


tions, no false-starts. 


Words you could use instead of “fail” include: iterate, 
adapt, adjust, course-correct, tweak, revise, morph, refac- 
tor, amend, fix. 


Success is not final, failure 
is not fatal: It is the 
courage to continue that 
counts.” 

—Winston Churchill 


Major errors 

Have you made a major error, perhaps a complete mis- 
judgment, incorrect strategic decision, or a project where 
months of effort by dozens of people has proved useless? 
That is indeed a big deal, and perhaps you’re even justi- 
fied in saying “we failed.” Still, that doesn’t explain its 
nature, and doesn’t point towards a solution, which 


would be the constructive thing to do. 


Words you could use instead of “fail” include: Pivot, 
evolve, migrate, overhaul, revamp, transform, reconstruct, 
repurpose, reimagine, redesign, recreate, reinvent, reengi- 


neer, remodel, rebrand, realign. 


Success is not built on 
success. It’s built on failure. 
It’s built on frustration. 
Sometimes it’s built on 
catastrophe.” 


—Sumner Redstone 


There are moments when we’ve truly, irrevocably, fatally 
failed, and we should say so. If a company has to shut 
down, especially before it achieved some meaningful suc- 


cess, then okay, we failed. 


2 of 3 


Stop saying “fail” 


Otherwise, find a better way to describe what’s happen- 
ing, so we understand the situation better, and are point- 


ed in the direction of what we should do next. 


Success is stumbling from failure 
to failure with no loss of 
enthusiasm.” 

—Winston Churchill 


Printed from: A Smart Bear 
https://longform.asmartbear.com/fail/ 
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The fundamental forces of scale 


by Jason Cohen on September 24, 2023 


These forces make larger companies slower and more 
difficult to execute, but also more effective when harnessed 
and leveraged. 


“I’m not letting myself go, I’m scaling.” 


Idealistic founders believe they will break the mold when 
they start scaling. They will not turn into a “typical big 
company.” 


By which they mean: No stupid rules that assume em- 
ployees are dumb or evil, with everything taking ten 
times longer than it should, with wall-to-wall meetings, 


and hiring “average” or “normal” people. 


That is, preserving the positive characteristics of tiny or- 
ganizations while avoiding the common problems of 
large ones, by adapting the startup’s existing processes, 
that have served them so well thus far. We’re merely do- 
ing it with more people now, and still figuring it out as 


we go along, exactly as we always have. 


Why do they never succeed? Why is it impossible for all 
of these intelligent, well-meaning founders to run a 500- 
person organization like a 50-person one? What are these 


unavoidable forces of scale? 


Are they really unavoidable? 


The two fundamental challenges of scale 


After understanding these two primary drivers, we'll see 
how they explain the challenges that arise in every cor- 


ner of the business. 


Rare things become common 


At scale, rare things become common, as fully explored 
in that linked companion article. 


Rare things are difficult to predict, and typically difficult 
to prevent. Before scale, they are, well, rare, so this 
doesn’t matter; they can be handled manually. But when 
they start happening daily, new processes become 


necessary. 


For instance, with 1,000 servers, a server failure every 
three years translates to one failure per day. This fre- 
quency impacts not just DevOps but also customer sup- 
port and social media teams. Customers are impacted 
and complaining every day, even if each individual event 
is rare, impossible to predict, and impossible to prevent. 


Automation helps, but doesn’t solve the problem. 
This is inescapable math crops up all over the business. 


Complexity beyond human comprehension 
Human minds have limited computational abilities. A 


sufficiently complex system cannot be fully comprehend- 
ed at all. 


“Complicated” systems can be comprehended. These 
types of systems are intricate, but they exhibit properties 
which allow people to tame the beast. Components of 
complicated systems can be separated, built and tested 
on their own, then assembled; thus, divide-and-conquer 
is a useful technique. Challenges are difficult but are 
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solvable, especially if you have seen them before; thus, 
“experts” and “specialists” can solve the puzzles. 


Repeatable processes can monitor the system. 


“Complex” systems, in contrast, have components which 
affect each other, often cyclically. Therefore, while it’s 
still useful to build and test components separately, 
there’s just as much complexity in their interaction. 
Indeed, most of the difficulty lies in those interstitial spa- 
ces. The human brain is composed of neuron “compo- 
nents” which are relatively well-understood; it’s in the 
complex, multi-directional, cyclical interactions that 
everything interesting—and inscrutable—unfolds. Large- 
scale effects like “consciousness” completely evade our 
understanding, as it emerges from complexity we have 


yet to comprehend. 


Complex 


Enabling constraints 


Complicated 


Governing constraints 


Loosely coupled Tightly coupled 


probe-sense-respond 
Emergent Practice 


sense-analyse-respond 
Good Practice 


Chaotic 


Lacking constraint 
De-coupled 


Clear 


Tightly constrained 
No degrees of freedom 


act-sense-respond 
Novel Practice 


sense-categorise-respond 
Best Practice 


This terminology comes from the Cynefin Framework. 


As a company scales, there are both kinds of systems, but 
the complex ones are fundamentally difficult, and never 
stop being difficult. Their difficulty grows non-linearly as 


the company grows. 


And even our unthinkably complex human brains, cannot 
comprehend the system. 


With the two fundamental principles in hand, we can de- 


tail specific challenges of scale. 


The human cost of robust teams 


A “team of one” is the fastest, most efficient team, if you 
measure by “output per person.” There’s no communica- 
tion, no meetings, and decisions can be made instantly. 
Small companies operate this way by necessity, and it 
works! It’s a big reason why they move quickly, and 


“punch above their weight” as is often said. 


But, an illness takes the velocity of the product or the 
quality of customer service from heroic to zero. With a 
small team, if one person leaves, you’ve just lost six 
months to hiring and getting-up-to-speed on that project. 
Or twelve months or a complete rewrite because there 
wasn’t any processes and documentation in place... be- 
cause it was just one person, who didn’t need that stuff, 
because after all we’re moving so quickly! And not 


communicating! 


Or it’s fatal because that was a co-founder. “Founder 


trouble” is a leading cause of startup death! 


1 Although data also show that companies with only one founder 
are more likely to fail. So which is better? To me, it comes 
down to temperament—some people really want to be the only 
one at the top, and some want to share the burdens with 
others. 
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ANDERSON 
“I dunno, ‘first mate’ has been 
done to death. How about this -- 
‘Co-pirate?’”” 


So a team of one is fast, but brittle. When you’re small, 
this is a good trade-off, because speed is critical for com- 
bating the things that are constantly about to kill the 
company—lack of customers, lack of market attention, 
lack of core features, all the things. When you’re large, 
with 15-25% annual employee turnover, not to mention 
illness, vacation, and family, people will be leaving (tem- 
porarily or permanently) every day—rare things becom- 
ing common. Maintaining the same attitude as when you 
were tiny is disorganized and irresponsible, and the com- 


pany would fail to function. 


So, at scale, no project can have fewer than, say, three 
people dedicated to it, plus management and possibly 
some form of Product or Project Management. But that 
team of 4-5 will not be 4x-5x more productive than the 
one-person team; per-person productivity goes down 


in exchange for robustness and continuity. 


On the other hand, while the small company loses 9 
months to the loss of a key employee, or even implodes, 
the big company is the steady turtle that adds thousands 


of customer per month like clockwork and wins the race. 


The high cost of robust systems 


You might think that software systems wouldn’t suffer 
the same brittleness as human systems, because we have 
things like automation and DevOps. Sadly, robust soft- 
ware is similarly much more costly and complicated. 


Consider the case of serving a website (don’t worry, you 
don’t need to be an software engineer to follow this ex- 
ample!). Suppose there’s a network connection to the 
Internet, a server that runs the web site’s software, and a 
database that holds the web site’s content. We have a sin- 
gle server, and it works most of the time. Let’s say it 


works 99.9% of the time; that’s sounds pretty good! 


Well, it’s not that good, actually. Failing for 0.1% of a 
year means it has failed for more than 500 minutes— 
surely an unacceptable amount of down-time! This hap- 
pens because a rare thing (0.1% failure) became com- 
mon (at the scale of about half a million minutes in a 


year). 


What if we had two servers instead? That way, when one 
fails, the other is still available. They'd both have to fail 
simultaneously for the website to be offline, which would 
happen 0.1% x 0.1% = 0.00013 of the time, which 
would be only 30 seconds per year, so that really is 


great~ 


OK, that’s twice as expensive, because we now have two 
servers, but it’s worth it for being more robust®. But wait, 
that’s not enough, because the traffic that comes in from 
the Internet needs to be distributed across the two 
servers, and it needs to have some intelligence to route 
traffic to the healthy one, in case one is failing. That’s 
called a “Load Balancer,” and all cloud infrastructure 
providers have them, and they cost more money. So that’s 


another component to manage, and more expense. 


2 Sadly this is not really how the world works; typically failures 
are correlated or cascade. We'll keep things simple for sake of 
the example, but this complication proves the point yet again. 
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3 You might think we can save money by buying cheaper servers, 
because each needs only one half the capacity of the original. 
But no, because in the failure mode, a single server still needs 
to serve all the traffic, so each needs to be powerful enough to 
work on its own. 


And wait, there’s also that database. The two servers 
need to share a common database of content, so the 
database needs to be moved off to its own server. And 
what if that server fails? Then we have downtime again. 


So we also need two of those for continuity. 


Now we're up to four servers plus a Load Balancer, and 
more than 4x the original cost. You can buy robustness, 
but it’s far more expensive than you might think. We 
haven’t even gotten to the fact that managing multiple 
servers is more difficult, or that there can be failures in 
the communications between servers, or failures in the 
Load Balancer which takes down the whole system*—all 
decreasing our robustness, requiring even more effort to 


deliver the end result. 


4 Yes, there’s solutions to this too—more redundancy at the net- 
work layer, often with a different vendor, but again, all more 
cost and complication! 


Robustness always carries a cost. Whether it’s running 
four servers instead of one, or a team of four instead of 
one, you trade local inefficiency for whole-company 
continuity. 


Predictability (that you actually need) 


When yov’re small there’s no need to predict when a fea- 
ture will ship. Marketing isn’t scheduling a launch and 
Recruiting isn’t timing the start-dates of the next 50 hires 
in customer service and sales. This means you can—and 


should!—optimize myopically for speed-to-market. 


Small companies correctly brag about their speed as an 
advantage, but it’s easy to see how a larger company has 
a different—and massive—advantage of execution. Sure, 
when our company WP Engine launches a new product, 
the marketing department needs predictability for the 
launch date, but that’s because it’s a highly-skilled, well- 


funded, coordinate group of teams that explode with 


press, events, campaigns, social media, and newsletters, 
grabbing more attention in a single week than a smaller 
company might scrape together in a year. There are also 
global Sales and Support teams, so we’re immediately 
selling to 200,000 existing customers as well as thou- 
sands of new customers per month, which means we'll 
add more additional revenue each month than a small 


company will earn over a whole year. 


But all this requires predictability. We didn’t line up that 
press and have those sales materials and train hundreds 
of support reps and ensure that code-quality is high 
enough to scale on day one, without predictability. 
Predictability requires going slower. Predictability re- 
quires estimation (takes time), coordination (takes time), 
planning (takes time), documentation (takes time), and 
adjusting the plan across all teams when life inevitably 


unfolds differently from the prediction (takes time). 


1 1 vee. Se eet ee ee 


ANDERSON 
“AnderCorp surprised investors expecting the 
worst today by performing completely 
miserably.” 


When you're hiring lots of people, you need predictabili- 
ty. Consider the timeline of adding a technical support 
team member. First, Recruiting is casting about for poten- 
tial candidates. Then scheduling and performing inter- 
views. Then waiting for them to quit their job and take a 
week off. Then new-employee-orientation. Then class- 


room training. Then paired up with senior folks on the 
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floor as they ramp up their skills and comfort. Therefore 
the time between deciding to grow the support team, and 


having new people up-to-speed, is four to six months. 


Therefore, we have to predict the demand for technical 
support four to six months in advance, because we need 
to be hiring for that right now. If we under-estimate, our 
support folks get overwhelmed with too much work; 
their quality of life suffers, and service to each customer 
suffers. If we over-estimate, we have too many people, 
which is a cost penalty. Of course the latter is a better 
failure mode than the former, but both are sub-optimal, 


and the solution is predictability. 


“The future is inherently unpredictable,” insists the small 
company, spurred on by Lean and Agile mindsets, and 
the truth. Indeed, blue-sky invention and execution are 
unpredictable. But this is also a self-fulfilling prophecy; 
to insist the future is unpredictable is to ignore the work 
that could make it more predictable, which makes it in 


fact unpredictable to that person. 


ANDERSON 


“But, to be fair, there's a fifty-percent chance 
of just about anything.” 


Small companies don’t have the data, customers, institu- 
tional knowledge, expertise, and often the personal expe- 
rience and skillset to predict the future, so they are usual- 
ly correct in saying it’s impossible. But is it impossible in 


principle, or is it impossible for them? At scale, it be- 


comes required. Not because Wall Street demands it, nor 
because investors demand it, nor any other causal 
derogatory excuse made by unpredictable organizations, 


but because it’s critical for healthy scaling. 


The Materiality Threshold 


If Google launches a new product that generates 
$10,000,000/year in revenue, is that good? No, it’s a fail- 
ure. They could have taken the tens of millions of dollars 
that the product cost to develop, used that to make their 
existing operation just 0.01% more effective, and made 


far more money. 


At more than $100B/year in revenue’, Google can only 
consider products which have the potential to generate 
$1B/year in revenue as an absolute floor, with the poten- 
tial to grow to $10B/year if things go better than expect- 
ed. Things like YouTube, Cloud, and self-driving cars. 


5 Editor’s note: This was originally written in 2017; in 2023 
Google’s revenue is nearly $300B, which reinforces the original 
point; they could not have achieved that growth if they worked 
on projects that generated revenues of “merely” $100M. 


This principle is called the “Materiality Threshold,” i.e. 
the minimum contribution a project must deliver, for the 


project to materially affect the business. 


With a small business, the materiality threshold is near 
$0. A new feature that helps you land just a few new cus- 
tomers this month is worth doing. A marketing campaign 
that adds two sign-ups/week is a success. Almost any- 
thing you do, counts. That’s nice—it feels good to be 


moving forward. 


The financial success of the larger company dictates a 
non-trivial materiality threshold. This is difficult. Even a 
modest-sized company will need millions in revenue 
from new products, maybe tens of millions in the opti- 
mistic case. Very few products can generate that sort of 
revenue, whether invented by nimble, innovative startups 


or stately mature companies. As proof, consider that the 
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vast majority of startups never reach a $10M/year run- 
rate, even with decent products and extraordinarily dedi- 


cated and capable teams. 


Yet, it’s the job of a Product Manager at that mid-sized 
company to invent, discover, design, implement, and nur- 
ture exactly those products—something that most entre- 


preneurs will never succeed at. Tough job! 


Recruiting 


Employee #2 will join a startup for the experience. Even 


with a significant salary cut, and even if the company 
fails—the most likely outcome—it’s worth it for the sto- 
ries, the influence, the potential, the thrill, the control, 


the camaraderie, the cocktail-party-talk. 


Employee #200 won’t join for those reasons. Employee 
#200 will have a different risk-profile regarding their life 
and career. Employee #200 will be interested in different 
sorts of problems to solve, like the ones listed in this arti- 
cle instead of the ones where you're trying to understand 
why 7 people bought the software but the next 3 didn’t. 
Employee #200 will not work for a pay-cut. 


“ee, Ley 
le are 


“So, where do you see yourself in ten minutes?” 


Small companies view this as an advantage, and certainly 
it’s advantageous to recruit amazing people at sub-mar- 
ket rates. But there hundreds of employees at WP Engine 
today who are much more skilled in their area of exper- 
tise than I’ve ever met at a small startup, including my 


own. Why? 


One reason is that people with a lot of experience are of- 
ten in a different phase of life, where family and other 
demands means they want a predictable, larger paycheck 
for a predictable, well-defined job. When you combine 
their advanced skills with the raw materials of a scaled 
company (i.e. customers, brand, funding, teams that can 
execute larger ideas), they can generate huge value while 
still tucking their kids into bed at night. 


Another reason is that, after developing that expertise, 
they find it’s enjoyable to apply their skills within a larger 
environment. For example, there are advanced marketing 
techniques that would never make sense with a smaller 
company, that are fascinating, challenging, and impactful 
to the top line at a larger company. There are talented 
people who love that challenge and would hate going 
“back to Kindergarten” as Jeff Bezos famously quipped, 
scratching out an AdWords campaign with a $2000/mo 
budget or assembling the rudiments of SEO or just trying 


to get a single marketing channel to work or being called 


a “growth hacker” because they finagled a one-time 


bump in traffic. 


This has implications on compensation, how you find 
that talent, and why that person wants to work at your 
company instead of the one down the block who can pay 
a little bit more. Therefore, it’s critical to have a mission 
that is genuinely important, have meaningful and inter- 


esting work to do, connect everyone’s work to something 


bigger than any of us. These matter even more at scale, 


because it’s the reason why talent will join and stay. 


“I don’t get it! We've got a mission statement, 
a credo, and a mantra!” 
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Communication 


With four people in a company, any information that 
everyone needs to know can be told to just three other 
people. Everyone can know everything. If there’s a 5% 
chance of significant misunderstanding, that doesn’t hap- 


pen often. 


With four hundred people, it’s never true that a piece of 
information can be reliably communicated, in a short pe- 
riod of time. A 5% chance of misunderstanding means 
twenty people are confused. That’s assuming they even 
read the communication, or listened during the entire 
presentation over Zoom. What’s the chance of that? Don’t 


ask. 


“Slack” is not the answer. “Email” is not the answer. The 


answer is: Repeating simple messages. 


“Repetition” is the answer to “I didn’t see it.” Repetition 
in different formats, at different times, by many leaders. 
Asymptotically achieving 100%, although also creating 
collateral damage for the people who really do read and 
listen to everything, who are tired of hearing the same 
thing, and wondering why theyre being punished for ac- 
tually paying attention. 


“Simple” is the answer to “I didn’t understand / remem- 
ber it.” Just as with simplicity in strategy, you have to ac- 
cept that people don’t read, people don’t remember, peo- 
ple have other things on their minds, people don’t under- 
stand language as well as you wish, and you didn’t write 


as clearly as you hoped. 


But doesn’t this mean you cannot communicate anything 
complex to 1000 people? Yes, that’s what it means. Scale 
is hard. 


Technology & Infrastructure 


Managing 10,000 virtual servers in the Cloud Era sounds 
easy. Automate everything, then any process that works 
for 100 servers, will work for 10,000 servers just by do- 
ing the same thing repeatedly—exactly the thing comput- 


ers are excellent at. 


It never works like that. Reddit took 18 months to get 
“number of likes” to work at scale. StackOverflow took 4 
years to get everything converted to HTTPS. Wired did 
that conversion in a “mere” 18 months. Everything is 


hard at scale. 
What are the patterns in those stories? 


One is another application of “rare things are common.” 
Rare things are hard to predict and can be hard to pre- 
vent. Often they’re hard to even identify and sometimes 


impossible to reproduce. This is fundamentally difficult. 


Another is continuity or compatibility with existing tech- 
nology. New companies get to start from scratch, but at- 
scale companies must transform. New companies like to 
make fun of large companies for how hard it is to trans- 
form, neglecting that the cause of the difficulty might 


also be generating $100,000,000 in revenue this year® 


5 Instead of “legacy code” we call it the “revenue code.“ 


Another is bottlenecking. All hardware and software sys- 
tems have bottlenecks. At small scale, you don’t run into 
any bottlenecks, or at least the ones you do can be solved 
with simple techniques like increasing capacity. 
Eventually something difficult breaks and you have to 
rearchitect the stack to solve it. Even something simple 
like converting HTTP links to HTTPS or updating “num- 
ber of likes” in real-time, becomes a monumental archi- 


tectural challenge. 


Another is Hickam’s Dictum: Problems in complex sys- 
tems generally have more than one root cause. Consider 
the case of a software upgrade causing problems for sev- 
en customers, in a way that wasn’t detected. What is the 
“root cause” of this problem? 


e The problem is we didn’t detect the failure; had we 
done so, we could have reversed the upgrade 
immediately. 

e The problem is we didn’t test the thing that failed; 
had we had more tests, the problem would never 
have reached customers. 
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e The problem is we didn’t document the code that 
broke; had the code been clearer, the human 
wouldn’t have coded the error. 

¢ The problem is we didn’t review the code properly; 
had the code been reviewed with a proper checklist, 
we would have discovered the bad code before it was 


deployed. 


As with Five Whys, there are always deeper reasons, as 
well as different reasons. Different from Five Whys, there 
isn’t just one that is the “root” cause; instead there are a 
lot of things that interact with a lot of things. This 
doesn’t mean it’s hopeless, it just means it’s complex to 
analyze, complex to decide what to do, and that Occam’s 


Razor doesn’t apply. 


All this slows down development and adds investment. 
There will be entire teams who focus on infrastructure, 
scaling, deploys, cost-management, development process- 
es, and so forth, none of which are directly visible to or 
driven by the customer, but which are necessary to man- 


age the complexities of scale. 


Risk-mitigation 


For a small company, the most likely cause of death is 
suicide. Usually it’s starvation—can’t get enough cus- 
tomers (distribution) to pay enough money for long 
enough (product), in the cascade of things-that-must-be- 
true for product/market fit. But also things like founders 
splitting up, not getting enough traction to self-fund or to 
secure the next round of financing, having to go back to 


a day job, and so on. 


At scale, the risks are different. There is very low risk 
that WP Engine will not sign up thousands of new cus- 
tomers this month. Other risks, however, are not only 
possible, but likely. Addressing those risks head-on, is re- 
quired for a healthy and sustainable business that can 


last for many years. 


Take the risk of business continuity during a disaster sce- 
nario. What if an entire Amazon datacenter became dis- 
abled for a week? How quickly could we get all those 


customers back up and running? Would that be true even 
though thousands of other businesses are also trying to 
spin up servers in other Amazon data centers at the same 
time? Could we communicate all this with our customers 
quickly and simply, so that our support team isn’t over- 
whelmed by repeating the same message to tens of thou- 


sands of justifiably-angry customers? 


"5 
ANDERSON 


“It’s important to remember that correlation does not 
imply causation. Besides, we all know it was Brian.” 


Risk-mitigation can even result in growth. Serious cus- 
tomers want to work with vendors who understand and 
mitigate risk; this maturity becomes a selling point. 
That’s why enterprise suppliers (like WP Engine) are con- 
stantly flouting their compliance with SOC 2 and ISO 
27001 and all the others. Small companies make fun of 
those things as being unnecessary at best or a false sense 
of security at worst, but while they're busy making that 
point, the larger companies are busy signing three-year 
multi-million dollar deals. 


Early on, you do not need a disaster-recovery plan. That 
won't be the thing that will kill the business, and your 
customers will understand if a young business is subject 
to that sort of risk. Later on, this becomes critical, and 


worth investing in. 


Large companies are not just small 
companies x 100 


These forces cause larger companies to be fundamentally 
different than small ones. It isn’t a bad thing or a good 


thing. It’s a different thing. 
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Some idealistic founders believe the root cause of scaling 
issues is the “command-and-control” organizational 
structure. But none of the examples above make refer- 
ence to any organizational structure. It’s universal. This is 
why Holacracy and Teal Organizations do not solve these 
problems. It could be a fantastic idea to experiment with 
organizational structure, but the fundamental forces 
above will not be eliminated with a recombination of 


roles and power structures. 


Scaling is hard, the road is foggy and bendy, it lasts for 
years, the set of people you need might be different, and 
no one emerges unscathed. So, it is not a sign of disaster 
if you have difficulty wrestling with these forces. 


Everyone does. 


Disaster is when a company is scaling, but the leaders 
don’t appreciate these forces, don’t work constantly to 
morph the organization accordingly, don’t bring in expe- 
rienced talent, decide they can figure it all out as they go 
along without help. Rather, it should mean new people, 
new roles, new values, new processes, new recruiting, 


new stories, new constraints, new opportunities. 


Too many founders and leaders want to believe: 


What got us here is what’s important and unique about 
us, and thus we should preserve all of it. Other compa- 
nies fail because they “act like big companies,” but 
we'll avoid all that because we’re smarter than they 
were. As evidence of our acuity, just look at our success 
thus far. We will continue to succeed in the future as 
we have in the past. 


But they're wrong. 


There should be a few values that are kept constant, 
that’s true. Otherwise none of it means anything. But the 


details must change. 


Many founders and leaders can’t make the shift. This al- 
ways hurts the company, and sometimes kills the compa- 
ny. The world is full of those horror stories. It’s sad, be- 
cause it’s an avoidable waste of opportunity and some- 


times hundreds of person-years of effort. 


Don’t become one of those cautionary tales. 


Printed from: A Smart Bear 
https://longform.asmartbear.com/scale/ 
© 2007-2024 Jason Cohen W @asmartbear 
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Metrics that cannot even be measured in retrospect 


by Jason Cohen on October 22, 2023 


Some of the most enticing, important metrics are impossible 
to measure, even after the fact. Here’s how to identify and 
avoid this trap. 


“Where do I see myself in five years? 
Maybe a 5 or 6.” 


Some things cannot be measured, even after months pass 
and we have all the facts and data and analysis at our 
disposal. 


If they cannot be measured in retrospect, they cannot be 
measured now, and you certainly cannot predict them or 


make a goal from them. 


This sounds obvious, but many of these are so tempting, 
seemingly so important, we end up trying anyway. But 


it’s a frustrating waste of time, and we ultimately fail. 


Here’s some common examples so you can train your pat- 
tern-matching engine, and see how to navigate the 


conversations. 


Impact of a single feature on the revenue 
of a product 


There are features at WP Engine which sales team pitch 
because people respond with genuine excitement. But 


then, after the sale, customers rarely use them! Does 


that lack of usage mean the feature isn’t contributing to 


revenue? No, it might have helped the sale 


1 Don’t worry, we don’t charge extra for those features, so it’s not 
unethical. 


2 PS. Do you think it’s wise for our competitors to copy those 
features? This is why you need to listen to customers instead of 
copying your competitors; who knows what’s going on over 
there. 


A feature that is used frequently is almost surely valu- 
able; it’s reasonable to claim that this feature is one of 
the main reasons why people pay for the product. But 
how much influence is it really? What if ten other fea- 
tures are also used frequently; do each “earn” 10% of the 
revenue? What if the feature is also used frequently by 
people who cancel—that would indicate the feature 
wasn't keeping them here after all. But maybe it was, and 
something totally unrelated caused the cancellation. 
What about all those other reasons customers are willing 
to pay that are unrelated to that oft-used feature? 


This is too difficult to unravel. This is why Llike using a 
variety of KPIs, only one of which is “usage.” Of course 
product teams should celebrate when they create fea- 
tures that are widely used. It’s just not the same thing as 


revenue. 


Even after the fact, with 100s of 1000s of customers (in 
our case at WP Engine), it’s still not possible to attribute 


revenue to individual features. 


Impact of incremental activities on 
customer churn 


As covered in another article, customer churn severely 
limits growth and, more importantly, is a lagging indica- 
tor that customers don’t want what you’re selling. 


Therefore, it’s almost always a good use of time to uncov- 
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er the main drivers of churn, and address them with 
product, process, people, price, and other words that 


start with “p.” 


So the common and very reasonable question is: If we do 
X, how much will churn go down? And while we’re actu- 
ally doing X, how much is churn going down? And this 
leads to the crux: A year from now, will you then know 
the impact X had on churn? Unless X has an enormous 


and immediate impact, the answer is no. 


The reasons often apply in cases besides churn, so let’s 
list them separately to train your pattern-matching en- 


gine, while continuing to use churn as the example: 


Cannot measure rare things, because signal is 
swamped by noise 

If churn is 3%/mo, an initiative that reduces churn by 
10%—a big impact!—will result in 2.7%/mo. How 
hard is it to measure a difference of 0.3%, month over 
month? How much does churn vary through pure ran- 
domness? Probably more than that. Some months 
have 15% more weekdays than subsequent months; if 
most customers churns on weekdays, that could make 
churn vary by 15% for that reason alone—again, pure 
noise. You probably just can’t measure something that 


is so rare. 


Lag between action and reaction 

A customer who churns today has probably been un- 
successful for a while. They’ve been researching alter- 
natives for weeks or months, weighing whether to pull 
the trigger. Larger customers might have tried to make 
it work for a year or more, then ran a multi-month 
process to investigate the options. So, an activity you 
start today is unlikely to change the trajectory of cus- 
tomers who have already decided to leave, and only 


today happened to push the red button. 


Many causes of a result, means it’s hard to mea- 
sure a change in any one cause 

There are many reasons why people leave. Some were 
never that interested, and didn’t get hooked. Some got 
mad because of support interactions, even though 


they like the product. Some finally gave up waiting for 
you to make a feature they've needed, even though 
they love the support interactions. Some were never 
the right fit, but tried to make it work, and so did you. 
Some forgot they even had it unless their spouse 
asked what this credit card charge is. So any one ac- 
tion you take is likely incremental, difficult to measure 
in the fog of almost every customer not caring about 


that particular thing. 


External forces that are larger than forces under 
your control 

The market, competition, and the economy at large 
has a large yet unmeasurable impact. You can just go 
blaming the global economy—you have a business to 
run, and you are in control of more than you’re cur- 
rently controlling. Still, it’s easy to map things like 
growth rates and churn rates onto the US stock mar- 
ket or inflation. Sometimes it can be the largest factor. 
How, in the face of these major factors, will you mea- 
sure the tiny blips that your incremental work might 


accomplish? 


Does this mean we shouldn’t work on improving churn? 
Of course not—again it’s one of the most valuable things 
any company can do, because retention compounds, and 
all the profit generated by a customer of a SaaS business 
happens in the later years, after they’ve paid back the 
sales and marketing expenses, gotten past the higher- 
than-average initial support load, and after your gross 
margins expand with scale. 


But, trying to measure the impact of a single activity is 
folly?; even a year later, you'll never know how much 


that factored in relative to all the other things. 


3 The exception is when your churn is especially bad; anything 
over 3%/mo is scary. Then, sometimes it’s possible to make 
large improvements. 
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Probabilities of “risks” 


The “Risk Slide” lists things that could derail the project 
or even tank the company. Executives quite rightly want 
to know: What is the chance that this will happen, and 
what is the magnitude of the impact if it does? 


We can’t invest in mitigating low-probability risks, al- 
though we might monitor the situation if the impact 
would be high. Conversely, high-probability risks might 
need proactive mitigation efforts. Thus the “probability of 
the event” is critical information, because it determines 
our actions. 


But these probabilities aren’t possible to measure, even 


after the fact. 


Each risk is independent. It will happen, or it won’t. You 
won't find out for months, maybe years. Then one or two 
happen and six don’t. Maybe the one that happened was 
listed as a 30% probability*, and one that didn’t was list- 
ed at 70%. Does that mean the probabilities were wrong? 
No, because 30% doesn’t mean 0% and 70% doesn’t 


mean 100%. 


4 if you're lucky; I often see probabilities listed as “medium” or 
“low.” Here’s how to talk about probabilities properly. 


This is similar to weather forecasting. If there’s a 30% 
chance of rain, and it rains, was the forecast correct? 
While you cannot answer that for a single day, there is an 
answer across hundreds of days: You take all of the days 
for which the forecast was 30%, and see how often it did 
rain. If the real result is, say, 34%, the forecaster is doing 
a great job; if the real number is 70%, the forecaster 


needs improvement”. 


5 Actually that’s not the full story; here is the fascinating math 
behind evaluating forecasts. 


ANDERSON 


“But, to be fair, there's a fifty-percent chance 
of just about anything.” 


That’s great, but not for our Risk Slide. We don’t have 
hundreds of measurements from forecasts of well-defined 
events. We have a slide of a handful of independent, 
complex risks, none of which have ever happened before. 
So we can’t know whether our probabilities were correct, 
even years later when we know with complete certainty 


what actually happened. 


So don’t put probabilities on the slide at all. Only list the 
risks that you feel are so important that they either merit 


action or awareness. 


More examples 


Now that the general pattern is clear, here are more ex- 


amples to avoid: 


Measuring the effect of small design choices on 
user experience 

Things like color, typography, layout, and word-choice, 
definitely matter, but typically noise overwhelms sig- 
nal in attempting to measure it, especially in small it- 
erations, and it’s not effective to survey users about 
exactly which font they like better. Also, any UI 
change initially results in lower satisfaction scores, be- 


cause people don’t like change. 
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Measuring the effect or ROI of company culture or 
values 

Having a strong culture, with a specific direction, 
helps you create a healthy organization of people who 
are alike in the ways that matter, while creating diver- 
sity in all other ways. But, measuring the impact of a 
specific value or a specific culture-related initiative, 
doesn’t make sense. 


Measuring the value of brand 

While there are some techniques that could make 
sense for the largest companies in the world (e.g. “un- 
aided awareness” or “brand recall”), you generally 
cannot look at winning specific deals, or competitive- 
ness, or growth, and answer “How much of that was 
due to brand?” This includes marketing campaigns 
that are primarily about brand, such as company- 


sponsored events. 


Measuring the value of proactively fixing bugs or 
good engineering practices 

You cannot measure the impact of a bug that never 
shipped, or the impact of an engineering practice that 


is simply how you work. There is no counter-factual. 


Measuring “willingness-to-pay” or “value 


delivered” 

both to profits and genuinely happy customers, yet di- 
rectly measuring “value” or “willingness” is elusive. 
Even if you raise prices, and measure the change in 
the number of new customers or the ire and cancella- 
tion of existing customers, that still doesn’t measure 
“value” or “willingness.” Anyway, that’s a test you can- 
not run often, and a test that’s often destructive for 


brand and customer relationships. 


Measuring the efficacy of employee compensation 
or retention programs 

Overall retention is easy and important to measure, 
but it is always multi-factored, and the rules can 
change over time. Thus this is like measuring the im- 


pact of a feature on product retention. 


So stick with KPIs that measure specific aspects of your 
systems, and stay clear of things you cannot measure 


even after the fact. 
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Product/Market Fit: Experience & Data 


by Jason Cohen on November 5, 2023 


Companies that achieve Product/Market Fit — both self- 
funded and VC-funded - exhibit the same prototypical 
metrics curves and subjective experiences. 


“I had the ‘winner, winner’ dream again.” 


Every day people debate how to define “Product/Market 
Fit.” 


It’s best to start with the subjective experience, because 
there is no doubt when you have it. (If you’re not sure 


whether you have Product/Market Fit, you don’t.) 


If every day it’s a struggle to get customers, it’s not working 


yet. 


If every day it’s a struggle to keep up with demand, it’s 


working. 


It is a momentum change from “push” to “pull”—a 
change from fighting for each customer, one at a time, to 
a flood of signups that you can’t even explain. It’s a 
change from “how do we get more customers” to “how 


do we handle the influx of demand?” 


It’s working. Something clicked, like a final puzzle piece 
snapping into place, a “fit,” and suddenly the floodgates 


are open. 


It’s not all good. The avalanche of customer complaints 
outstrip your ability to deliver bug fixes and simple en- 
hancements. You feel behind in every department simul- 
taneously; you can’t keep up with the work no matter 
how “productive” you are. You obviously should hire 
someone—and this new revenue means you can afford it 
—but who? Someone just like you, because you can man- 
age them, or someone who complements you, so you can 
deal with the burgeoning scale? Does it even matter— 
you don’t have time to interview. 


It’s the proverbial “good problem to have,” but it doesn’t 
feel good in the moment, with a crushing workload and 
unhappy new customers, and your brand slipping away 
(“I heard they were good but it took them a day to an- 
swer my support question” says the public review). 
Although of course it is the best problem to have. 


You might think all this doesn’t matter—you’re just run- 
ning your company; who cares how some article defines 
some term like “Product/Market Fit?” But it does matter, 
because it determines whether the company is sustain- 
able, and fundamentally changes how you operate each 
day, how you plan for the future, whether and who you 
need to hire, and what you need to build, and whether 


youre going to start tackling scale. 


What does Product/Market Fit look like numerically? 


In my experience building several companies myself and 
angel-investing in dozens of others (with a wide range of 
outcomes), I define Product/Market Fit as all of the 
following: 
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1. Easy growth (pull, not push): Growth rate suddenly 
spikes up, and sustains at the new rate. You often 
don’t know why. 

2. High retention: Cancellation under 3%/mo (for 
B2B) or 5%/mo (for B2C), because if customers are 
leaving in droves, you are not a “fit.” 

3. Critical mass: At least $20,000/mo in revenue or 
increasing WAU’s by at least 200/mo. Everyone can 
grow 1000% month-of-month when the baseline is 7 
customers; the fact that you have only 7 customers 


means there’s not yet evidence of “fit 1” 


1 What if you have a single huge customer? Congratulations, lots 
of people would love to have a huge customer; great start. 
That demonstrates you have fit for a single customer, but not 
for a market; that’s consulting, not product. 


What follows are the data to back up this definition, that 
also matches the lived experience which is really the es- 
sence of hitting Product/Market Fit, where life will never 
be the same again. 


The characteristic growth curve of 
“Product/Market Fit” 


Bootstrapped products 


When you see growth charts like Pallyy’s, you can see the 


exact moment when Product/Market Fit was reached: 


$60,783 $387 $21 $31 $60,585 $125 $135 
Gime Poe ireacrvanion muveean meee a rove apie 


Pallyy’s four-year MRR 


Successful startups often have a growth curve like 
Pallyy’s: Revenue growing linearly, slowly, often for about 
two years, then in a moment—Product/Market fit—when 
it suddenly turns upward, and continues growing at that 


new trajectory. 


The new growth is still linear, but dramatically faster. In 
particular, it is not exponential, though that’s the word 


people frequently use (this universal fallacy is explained 


here). 


ConvertKit was similar, bumping along for two years be- 
fore growing quickly but linearly, just like Pallyy: 


$1,500,000 
$1,000,900 
$500,000 


$0 


Jan "13 Aug "13 Mar ‘14 Oct "14 May "15 Dec"15 Jul'16 Feb '17 Sep'77 Apr ‘18 Nov"18 dun"19 
ConvertKit’s MRR growth curve 


You can see the sudden change between “every sale is a 


struggle” and “the orders just won’t stop coming in.” 


ConvertKit founder Nathan Barry famously tells the story 
of how he personally reached out one by one to his ideal 
target customers, manually onboarding them to not only 
get the sale but to get the testimonial? It’s a brilliant 
playbook that others would be wise to copy. It’s also what 
it means to hard-scrabble for customers, and it’s really 
obvious when the forces flipping from having to scratch 
and claw for every dollar of revenue, to the company 


running away from him. 


2 I was one of them; perhaps you’re reading this article because 
of an email from my newsletter, which is still on ConvertKit. 


VC-funded products 

These companies were bootstrapped, but the it’s same in 
the world of venture, because Product/Market Fit is 
about products and markets and customers and fits, not 
about funding models. My company WP Engine started 
14 years ago self-funded but two years later became VC- 
funded, eventually raising more than $300M, experienc- 
ing hyper-growth® for a decade, yet it exhibits the same 
two-year slow-growth preamble to high-growth, and both 
linear: 
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3 Often defined as 2T3D, i.e. in successive years after attaining 
$1M in annual revenue, the company’s revenue triples, triples, 
doubles, doubles, and doubles, thereby getting close to $100M 
five years later; WP Engine did that and more, starting with 5x. 


2010 2011 2011 2012 2012 2013 2013 2014 2014 


WP Engine’s early growth curve 


You can’t tell from the chart when we first raised money 
(it wasn’t January of 2012), nor subsequent rounds, nor 
the size of the rounds. The shape of growth is a force that 


transcends all of it. 


I distinctly remember when we flipped to “Product/Mar- 
ket Fit.” Only five of us, completely sustainable, and sud- 
denly everything changed. We fell behind on our stellar 
tech support; one of the dangers of bragging about how 
“the founder does Zendesk tickets” is that process doesn’t 
scale. We couldn’t keep up with our free white-glove site- 
migration service for new customers. And then, com- 
pletely behind on all work, trying to stand up new 
servers and answer phone calls, knowing we needed to 
hire, we got even more behind because hiring takes time 
—finding candidates, interviewing, not wanting to hire 
the wrong person just because we’re moving fast, and 
even after we hired, there’s no documentation or train- 
ing, so we’re just working side by side, hoping folks learn 
by osmosis, while the team gets even further behind. And 
customers were complaining on Twitter that our service 
is going downhill—and worst of all, they were right. 
They blamed it on us raising money; what they didn’t 
know is we hadn’t spent a dollar of it yet; in fact our er- 
ror was not spending the money on hiring ahead of need, 


to blunt the impact of a growth increase. 


We did get back on top of it later in the year, and we’ve 
won all sorts of awards for our service, maintaining 98% 
CSAT to this day, because that’s who we are. But that’s 
not what it felt like in 2012. 


The same thing happens to the “cool kids” companies in 
Y¥-Combinator: 


Anonymous YC company growth chart from YC co-founder 
Paul Graham 


This isn’t a new phenomenon, nor a “SaaS” phenomenon. 
Peldi Guilizzoni famously shared his (one-time!) revenue 


for Balsamiq Mockups back in 2008: 
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Balsamiq Markups one-time (not SaaS!) revenue, 2008 
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Second products 

Hubspot is a hyper-growth VC-funded company, now 
public. Early on, you can see the linear behavior, fairly 
consistent even through the 2008 recession, as co- 
founder Dharmesh Shah indicates on a chart from his 


personal blog: 
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Hubspot’s early growth curve 


Sometimes there’s another shift in the slope of the line, 
when the company adds a second successful product, ex- 
pands to a lucrative new market, or changes its business 
model. Hubspot, after twelve years of linear growth, 
changes to a new (but still linear) growth rate around 
2021, around the time their second product-line reached 
its own fit: 


Quarterly Revenue 
($m) 


Hubspot’s at-scale growth curve, exhibiting a bend, and linear 
on both sides of the change 


Viral products 

All of the previous examples were of B2B companies 
without “viral” or “network” effects, which are often as- 
sociated with so-called “exponential growth.” However, 
even classic “viral growth” examples like Slack (at the 
time the fastest-ever-growing B2B company in history) 
and Facebook (at the time the fastest-ever-growing B2C 
company in history) did not grow exponentially. Instead, 
they grew in the same way again: linear, then a sudden 
change, then still linear but faster. Here’s Slack, which 


looks exactly like Hubspot: 


Slack daily active users 
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Slack’s early user growth curve 


And Facebook with a material bend in early 2009 (coin- 
ciding with the introduction of the “like” button): 
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Facebook’s early user growth curve 


Elsewhere, I’ve gone into great detail about why even vi- 
ral products like Slack and Facebook specifically don’t 


grow exponentially. 
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You can also see it in AirBnB with a bend in late 2010, 
coinciding with the realization that professional photog- 
raphy was the key to “fitting”? with customer 


expectations: 


10 million 
8 million 


6 million 


4 million Photographers 


2 million 
2008 2009 2010 2011 2012 


Airbnb’s final piece of the puzzle 


Media products 
While I do think it’s more usual for media and social me- 
dia to have a longer, slower curve, sometimes you see the 


Product/Market Fit pop, like Lenny Rachitsky’s news- 


letter: 


Lenny’s Newsletter traffic 


Oh, and what did he do to cause that sudden growth? 
You guessed it: 


Ali Attar .S @ @aliatttar- th 
What did you change in Jul 2022? 
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The bottom line is, these growth curves matching “fit” 
are common with all business models—B2B and B2C, re- 
curring-revenue or one-time revenue, products or media, 
self-funded or VC-funded. 


High cancellation means there’s no fit 


There is one killer metric that will stop growth in its 


tracks, even with the curves above: Cancellation. 


Founders often think cancellation of 5-7% per month is 
OK so long as they’re getting lots of new customers. But 


they are mistaken. 


High cancellation puts a ceiling on growth. This is on 
public display with Buffer, who started with the same 
sort of linear-change-linear growth curve (2012-2014, 
then 2014-2020), but then growth leveled off (2020) and 
never returned (and even shrunk): 


S15M 


$10M 


Jan 2012 Jan 2014 Jan 2016 Jan 2018 Jan 2020 Jan 2022 


Buffer’s ARR 


The reason is churn. Note how churn was higher during 
the slow-growth period of 2012-2014, then reduced to 
5%/month, permitting faster growth: 
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Jan 2012 Jan 2014 Jan 2016 Jan 2018 Jan 2020 Jan 2022 


Buffer’s monthly cancellation rate 


But 5% is still too high; there’s no way to keep increasing 
new-customer acquisition to outpace churn, for the sim- 
ple reason that “5%/mo” definitionally grows in lock-step 
with your total size, but marketing and sales does not 
continue to automatically grow just because your compa- 
ny is bigger. So, cancellations always win that race, and 
at 5%/mo, you top out pretty quickly: 


MEME New MRA ENN Expansion MRR MINN Reactivation MRR MINN Churned MRR [IN Contraction MRR. 
$300,000 


$200,000 


$100,000 
-$100,000 


-$200,000 
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Buffer’s unit economics; new-MRR naturally tops out, whereas 
cancellation never stops growing in absolute dollars. 


You can see a similar effect at SparkToro, an impressive 
company that is a testament to founder Rand Fishkin’s vi- 
sion to build a great company instead of a company that 
attempts to accelerate growth forever; he explains how 
SparkToro generates enviable profit, with happy employ- 
ees, happy customers, and happy investors. So this isn’t 
an indictment of the company, but it is a demonstration 
of the growth-destroying power of high cancellation 


rates: 


SparkToro is consistent in adding the same dollars-per-month 
of new-MRR, but their consistent cancellation rate continues to 
grow in dollars per month, which means at their current size, 
cancellation “wins” that race, and growth becomes flat. 


It’s not actually about finances; it’s the fact that cus- 
tomers are choosing to leave. The numbers are the ob- 
jective measure that this is happening, but they’re not the 
point. The point is that customers don’t want to stay. 


That means it’s not a fit. 


Therefore, both because “lots of customers leaving means 
its not a fit” and because high cancellations means the 
growth curve will quickly flatten, cancellation higher 
than 3% for B2B or 5% for B2C indicates a lack of 
Product/Market Fit. 


Easy inbound growth is indeed a sign of fit, but if cus- 
tomers leave in droves, it means the promise was right, 
the price was right, but the product didn’t deliver, or the 
customer’s need was too brief to sustain a recurring-rev- 


enue business. 


Some people call this “a fit, but not sustainable” or “a fit, 
just with limited upside.” I call it a lack of fit, because if 
customers are leaving, it’s not a fit* Don’t get distracted 


by dollars. Pay attention to customer behavior. 


4 See this amazing five-minute segment from founder of Twitch 
Michael Seibel, describing how his Socialcam app got 60M 
downloads in 4 months and top-5 on the App Store, yet reten- 
tion was so horrible (almost zero retained users after 10 days) 
that despite that inconceivable growth, it was not 
Product/Market Fit. 


If your company is growing slowly, that doesn’t mean you 
failed. It just means you haven't hit that success curve 


yet, and haven’t found Product/Market Fit. 
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Maybe you never will, but you'll grind out a profitable No one knows; that’s life. But now you know what it 
company anyway. Maybe you never will, and growth is so looks like, when it is truly a fit. 
slow, it’s best to face facts, stop, figure out why it was too 


hard, and try something else. Or maybe your growth 
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Quarterly strategic planning using the fairytale structure 


by Jason Cohen on November 26, 2023 


Traditional fairytale structure fits naturally in our brains, and 
thus can guide strategic problem-analysis, and a plan that 
everyone understands. 


“You ate some porridge, broke a chair, the bed thing’s 
a little weird, but I’m thinking 3 months community 
service, maybe a fine. You should see my other case. 
Two kids chow down on a witch's house, 
then burn her alive in her own oven.” 


Using the narrative structure of the fairytale, we can exe- 
cute the classic “Double-Diamond” design pattern, dri- 
ving our analysis, decision-making, actions, and our final 
exposition, in an intelligent, systematic way that every- 


one can easily understand. 
It’s going to look like this: 


1. Objectives: What are we trying to get / what 
happens if we don’t? 

2. Obstacles: What stands in our way? 

3. Actions: How do we overcome the obstacles while 
advancing our objectives? 

4. Measures: How will we know we've been successful? 

5. Not Doing: What important things have we chosen 


not to do, so that we can achieve the above? 


Now let’s tap into humanity’s collective unconscious, and 


see exactly how to run this framework in practice. 


The Fairytale cheat code 


The fairytale structure is thousands of years old, crossing 
all peoples of the globe, a reflection of a deep, shared 


humanity. 


You’ve heard it your entire life, from centuries-old child- 
hood stories to every Pixar film to advertising to the 


news, so you need only a reminder, not an explanation: 


. Once upon a time there was an unsuspecting hero. 

. Every day, the way things are. 

. But then, one day, change/event that motivates action. 
. And so adventure begins. 


. It was almost impossible because challenges/obstacles. 


. Until finally victory. 


N DOW BR WD 


. And forever after, the world is different. 


Fairytale structure is a cheat code for effective com- 
munication, as journalists have exploited for years. For 
example, this simple “case study” that I’ve used in sales 
calls at WP Engine takes less than a minute to tell: 


One of our customers is . Like you, they are a 
popular food blogger, making money through advertis- 
ing and occasional affiliate product sales. One 
Thanksgiving, on their busiest day of the year, their site 
crashed because of all the traffic; they missed out on 
thousands of dollars of revenue and hundreds of news- 
letter sign-ups. They moved their site to us, and that 
never happened again; they just focus on their work, 
build their brand and business, and never worry about 


technology. 


It’s equally effective for presentations, fundraising, and 
product proposals—a ready-made outline, resulting in a 
straightforward narrative structure. Our brains are pre- 
wired for fairy-tales, so we naturally follow the presen- 
tation as it flows into those primordial ruts in our gray 


matter. 
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A less obvious but extremely powerful application of this 
principle is in creating a strategic plan, i.e. analyzing 
the current situation and deciding what needs to be 
done. For a single team, this means being in command of 
the most critical things that need to be done right now. 
For a department at a larger company like WP Engine, 


this is how I build our quarterly strategic plans‘. 


1 It looks like ’m just slapping the word “strategic” on every- 
thing, to make it sound important. Indeed, people often do 
that. As you'll see, the purpose of this is to connect work to the 
strategy, so it really is strategic planning! 


Over the past two years, together with a few product 
teams at WP Engine, we’ve created and honed a simple 
but effective framework for the Fairytale Plan—strategic 
work-planning that has worked well both to drive the 
process of figuring out what the plan should even be, as 
well as communicating the result throughout the compa- 
ny. The process is akin to the widely-admired Double- 
Diamond design process, and the final result matches the 


fairytale structure. 
No wonder it works, and resonates with everyone. 


Here’s how you can do this too. 


Once upon 
a time... 


“Listen, I understand about the burden of exposition, 
but can we just see the 4th quarter numbers?” 


The narrative structure of the strategic 
Fairytale Plan 


Here’s how our strategic plan will look, and the guiding 


questions that help us create each section. 


Objectives 


Obstacles 


Actions 


Measures 


Not Doing 


Strategic Plan 
Narrative 
We must advance 
our strategy by 
achieving these ob- 
jectives. 


Fairytale 

1. Once upon a 
time... 

2. Every day... 


But obstacles stand 
in our way. 


Fairytale 

3. But then, one 
day... 

5. Obstacles” 


So here’s what 
we're doing. 


Fairytale 

4. Adventure be- 
gins... 

5. Tackling 
obstacles... 


And here’s how we'll 
know whether 
we’re making 
progress, and when 
we're finished. 


Fairytale 

6. Until victory... 
7. The world anew. 
Important topics 
and ideas that 
we’re not doing 
(yet). 


Guiding Questions 
¢ What is happening now? 
¢ What do we want? 
¢ What happens if we 
don’t get it? 
¢ If we weren’t encum- 
bered by roadblocks, with 
no distractions and no in- 
terruptions, how would 
we advance our strategy 


right now? 


¢ What is standing in the 
way” of “what we want?” 
¢ What makes it difficult 


to achieve the objectives? 


* How do we overcome 
the obstacles... 

¢ ...while advancing the 
objectives? 

¢ ...with the people and 
capabilities and strengths 
we currently have? 

¢ What changes need to 
be made, so that the an- 
swer becomes obvious? 


¢ How will our company 
be different? 

¢ How will we know, 
objectively? 


¢ What do we really want 
to do, but have to wait so 
that even more important 
or urgent things can be 
done? 

¢ What did we leave out 


that is contraversial? 


2 of 9 


Quarterly strategic planning using the fairytale structure 


2Ina fairytale it’s fun to surprise the hero with obstacles. Our 
plan must do the opposite: Clarify the obstacles up front. 


3 There are many kinds of obstacles: internal execution chal- 
lenges, interruptions, dependencies, competitive pressure, mar- 
ket dynamics, just to name a few. 


There are techniques and pitfalls for each of these sec- 


tions. Here’s how to navigate it: 


Strategic Objectives 


Narrative Plot Point 
We must advance our strategy by achieving these 


objectives. 


Guiding question 
If we weren’t encumbered by roadblocks, with no dis- 
tractions and no interruptions, how would we advance 


our strategy right now? 


Why can’t we skip this section? 
If you don’t know your strategy, if you don’t know how 
you'll win, then even if you run very fast, it will be in 


the wrong direction. 


GOALS: 


Achieve results 
Realize objectives 
Finalize outcomes 
Accomplish aims 


“I think I see why we're not getting anywhere.” 


Objectives shouldn’t summarize the entire strategy; your 
strategy document already does that. Instead, link to that 
document for reference, and highlight 1-3 specific objec- 
tives as the subset of the strategy that make sense to ad- 


vance right now. 


“Right now” is the difference between this plan and your 
strategy. The strategy is the long-term vision and the 


main ways that you will achieve it. The plan is where you 


select the specific next steps that materially advance the 
strategy while achieving results that you need today. “Re- 
sults today” is typically revenue growth, cost reduction, 


or some internal transformation. 


Later in the process of creating the plan, when you're de- 
termining how to achieve these objectives while also 
dealing with the obstacles, you might get stuck, unable to 
invent anything that works. In that case, you might need 
to choose different objectives. Or you might decide that 
“eliminating obstacles” is all you can accomplish right 
now—unfortunate, but useful to declare if true. During 
the planning process you should expect and even encour- 
age fluidity between exploring the problem-space and so- 


lution-space. 


The specific objectives will evolve over time, but it’s OK if 
they’re the same for a few quarters in a row. Indeed, that 
gives the team stability and focus. How do you know if 
youre sticking with an objective for too long? Later you'll 
be asking the question “how will the world be different.” 
If there are good answers, then we’ll know whether we’re 
making progress, despite the objectives remaining the 


same. 


Obstacles 


Narrative Plot Point 


But obstacles stand in our way. 


Guiding question 
What is preventing us from making rapid progress on 
the objectives? 


Why can’t we skip this section? 

Often the most important business challenges appear 
here, not in the objectives. This is where you face diffi- 
cult truths about your current situation, which will en- 
sure the work you select will actually overcome them. 
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If I had an hour to solve a 
problem, I’d spend 55 minutes 
thinking about the problem and 5 
minutes thinking about 
solutions.” 

—Albert Einstein. 


I agree with Einstein (but then, who among us is in a po- 
sition to argue against Einstein?). This is the most impor- 


tant section, and often where you'll spend the most time. 


You’ll be tempted to ignore Einstein, but it is crucial 
that you do not. You'll want to jump right to the solu- 
tion. You’ll say you’ve already talked about this stuff, and 
you know what to do. You'll say you don’t want to rehash 


all our challenges; let’s cut to the action. 


I understand, I feel the same way. I want to activate on 
any reasonably-good idea right away. It’s wrong, because 
if yowre moving quickly in the wrong direction, you’re 


not making progress. 


Without fully exploring the problem-space, you’re not in 
a position to even know the goal of the solution-space. 
You'll just keep repeating mistakes, keep avoiding the dif- 


ficult truths, and you won’t improve. 


It’s important to generate many options first; only by ex- 
ploring the whole space will you uncover the things 
which are the most important to solve now. Your team 
will thank you for the clarity, not only about what they 
need to tackle, but what they’re allowed to leave alone, 


even though it’s on fire. 


Here are some types of obstacles you should explore, in 


no particular order: 


Intrinsic difficulty: “Why is this hard?” 
Executing the strategy might be intrinsically difficult, 
like building something we’ve never built before (thus 


maximally risky and uncertain), or having to create 


novel algorithms, or having to deploy infrastructure at 
scale, or pivots, rewriting, rebranding, or repositioning 


that will take intense effort and coordination. 


Team challenges: Humans are hard. 

Do we have the right skillsets on the team? The right 
experience levels? The right motivation? Are they 
burned-out? Is there enough people to do all the 
work? 


Competitive pressure 
Is a competitor winning business in a way that de- 
mands a response, even if it’s not directly related to 


our long-term strategy? 


Customer retention pressure 
Are customers leaving at high rates* and therefore we 
have to address that immediately even if it’s unrelated 


to our long-term strategy? 


4 My rule of thumb for “what is a ‘good’ cancellation rate” is: 
3%/mo for consumer, 2%/mo for small business, 1%/mo for 
enterprise. More on that here. 


Interruptions 

Are we dealing with interrupt-driven work instead of 
planned work, such that we’re unable to make enough 
progress on strategic work? Do we need to do work to 
reduce the frequency or magnitude of the interrup- 
tions, perhaps through automation or rejecting certain 


types of work? 


Too many bugs 

Are we failing on our basic promises to our customers 
of quality and performance, due to mounting bugs or 
other issues? Do we need to attack those now, even if 


they are unrelated to our long-term vision? 


Dependencies / coordination 

Is it impossible for us to execute or deploy or iterate, 
due to some other team or process or platform or ar- 
chitecture? Do we need special coordination of work 


in order to achieve our own goals? 
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Process bottlenecks 

Are our tools getting in our own way? Is our process 
preventing throughput? Not just nice-to-have improve- 
ments, but step-changes in our productivity, and likely 


also our happiness? 


Market evolution 

Are there trends that are making our current product 
or positioning less effective or less relevant? Does this 
even mean our strategy needs to change, to keep up 
with the changing external landscape? 


Narrow these down to 1-3 most critical issues. The guid- 
ing question is: Which are the few issues where, if we 
solved them or took a huge bite out of them, but allowed 
all others to fester without any progress whatsoever, we 
would still consider that to be a victory. Whereas if we al- 
lowed those few to go untreated, and completely solved 
five others, we would still be unable to achieve our 


strategic objectives? 


If that question is still hard to answer, then as a guideline 
I like the following order of precedence. That is, if you 
have a serious problem in some of these categories, I gen- 


erally pick whichever comes first: 


1. Retention (if customers are leaving, the business 
model and product isn’t working) 

2. People (must have the right team) 

3. Throughput (can’t do enough strategic work per 
month) 

. Growth (of the things you have direct control over) 

. Competition 

. Health of the code base 


N DO wf 


. Execution risk 


A perfect formulation of a 
problem is already half its 
solution.” 

—David Hilbert 


Main Activities 


Narrative Plot Point 


Here’s what we're doing. 


Guiding question 
How do we overcome or side-step the obstacles, while 
advancing the strategic objectives, with our current 


people and capacity? 


Why can’t we skip this section? 

If you don’t select a few, large, impactful things to do 
(“Rocks”), you will wile away your time chipping 
away at incremental things, believing that you’re mak- 
ing progress because you are “agile,” but in fact not 
tearing down the obstacles nor achieving the objec- 
tives quickly enough. 


“Listen, you’ve got so many you don’t know what to do, and I love 
the taste of children. Maybe we can help each other out.” 


While putting your finger on the crux of the challenges is 
often the most emotionally-difficult part, deciding what 
to do is the most intellectually-difficult. It’s hard to think 
of great ideas, and no framework or process magically 
causes you to be creative and insightful. 


As with obstacles, you should start in “generation mode,” 
coming up with as many ideas as possible, before switch- 
ing into “selection” mode, where you pick which ones to 
do. You should already have queues of ideas from around 
the company; if not, now is a good time to create some. 
If you're having trouble coming up with creative ideas, 


use Extreme Brainstorming to shake things up. 
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If you’re having trouble figuring out how to deal with the 
obstacles, consider the parable of Herbie (from The Goal 
by Eliyahu M. Goldratt). A group of hikers are climbing a 
mountain. One hiker—Herbie—is constantly falling be- 
hind, preventing the group from achieving their goal of 
reaching the summit. There are three ways to deal with 
Herbie: (1) Kick Herbie off the team; (2) Help Herbie get 
better; (3) Decide that you’re not going to achieve your 
goal. These options help you think of solutions; respec- 
tively: (1) Let that problem burn or ignore certain work 
that you’ve assumed “needs” to be done; (2) Do work 
that mitigates the problem; (3) Change your objectives to 


something you can actually achieve. 


Often the best solutions come from synthesizing several 
ideas. For example, often at WP Engine an idea that in- 
creases website performance for our customers also de- 
creases our costs, because it takes less CPU time or fewer 
bytes transferred over the internet to accomplish the 
same end result. This is another reason to use things like 
Extreme Brainstorming; even if an individual idea is 
bonkers, it might be synthesized with a practical idea 


into a great solution. 


— 
ee 


“OK, but if we work together... Whammo! 
Depth perception!” 


For the final selection process I recommend the Binstack 
method, because you’re picking one or just a few critical 


big things to do. Your objectives and obstacles act as nat- 


ural filters, immediately eliminating anything that 
doesn’t directly address those, but allowing yourself a 


few secondary prioritization dimensions. 


You might be tempted to use an ROI analysis, but this is 


the wrong framework. ROI is good for smaller activities, 
where the goal is to find the “best use of time.” For your 
most critical pieces of work, the goal is not to optimize 
your time, but rather to maximize your impact on the 


most critical things. 


work for a complete system that covers all kinds of work, 
including these most-critical activities and medium-sized 


work where ROI analysis is appropriate. 


A successful solution to a 
problem makes the 
problem appear to have 
been nonexistent in the 
first place.” 


—James Surowiecki 


Metrics & Indicators 


Narrative Plot Point 
Here’s how we'll know whether were making 


progress, and when we’re finished. 


Guiding question 
How will we know whether it’s working? What is 


changing? What did we learn? 


Why can’t we skip this section? 

It’s easier to avoid accountability, believing that hard 
work and high throughput is the same as having an 
impact. We must not let ourselves off the hook, but 
neither should be measure ourselves against metrics 
we don’t believe in, or insist that all important things 


can be boiled down to a number. 
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“OK, we've got three sides now. I think we’re making 
good progress.” 


The “Activities” are supposed to make progress on the ob- 
jectives, but how will we know if that’s actually happen- 
ing? They’re supposed to deflating the challenges, but are 
they really getting eliminated? 


In the ideal case, we should measure progress with num- 
bers that are objective, well-defined, measured daily, and 
have a clear “stopping-point” where we can declare victo- 
ry. But we all know that most things we can measure 
aren’t so ideal, and sometimes the most important things 
aren’t a number. 


To the extent that important things are numbers, use this 
KPI framework to decide what things should be mea- 


sured, and as a checklist of what a “good” metric is. 


For concepts that are not a number, capture their essence 
in writing. Make the language crisp and obvious not only 
inside the team but to outside stakeholders. Nowadays 
you can use AI to take a rambling audio description and 
turn it into a pithy statement. An example might be: “Be 
revered as the most innovative product in the category” 


or “Have the best social media content.” 


These statements can be surprisingly powerful despite 
not being connected to a number. Even when people dis- 
agree later whether we’re making progress against them, 
those are exactly the right sorts of debates to have, be- 
cause you're sorting out how things are going, and 
whether you should make a change. That’s the point of 


metrics. 


Not Doing 


Narrative Plot Point 


Here’s what we're not doing (yet). 


Guiding question 
What do we really want to do, but we just have to 
wait so that even more important or urgent things can 


be done? 


Why can’t we skip this section? 

Decisions are about saying “no” even more than say- 
ing “yes.” Without articulating the “no’s,” the team is 
less focused, the decisions are less clear, and stake- 
holders wonder whether you’re even aware of the 


problems that you chose not to tackle. 


The difference between successful 
people and really successful people 
is that really successful people say 
no to almost everything.” 

—wWarren Buffett 


Deciding to do a few things always means deciding not to 


do dozens of other things. 


“Well, not doing them now” we like to say, to put the lis- 
tener at ease. But actually, most of them are “not ever,” 
because we always have 10x more ideas than we have 
time to execute. That’s the lesson of every Jira backlog, 


feature-collection system, or prioritization spreadsheet. 


When you don’t articulate what we’re not doing, it can 
appear that you don’t even know those things exist. 
Perception is reality. Stakeholders or even people on the 
team believe you’re ignoring important things, when in 
fact you've selected certain important things. You need to 


say that out loud. 


It’s not just about proving yourself to an executive, it’s 
about reminding yourself and your own team, especially 


when a big customer complains about one of the things 
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you're putting off, and the pressure is on, the emotions 


are high, and you need the resolve to stand firm on “no.” 


You can do anything, but not 
everything.” 


—Derek Sivers 


Next quarter, when you're going through this process 
again, this is a great list to revisit. You’ll be surprised how 
many items seem less important now than before. And 
some might be more important; that just means that per- 


haps now is the time. 


At WP Engine this a simple bullet-point list, just one line 
per bullet. It says what the topic is, and doesn’t even ex- 
plain why it wasn’t selected. You could go further, ex- 
plaining why each didn’t make the cut. You could go still 


further, creating a “Now / Next / Later” roadmap” 


5 Or, my personal preference in that three-column genre: “Now / 
Likely / Plausible.” This better indicates not only timeline but 
our confidence that those are even the right ideas. After all, 
how many of those “Later” items do you ever get to? If the an- 
swer isn’t at least 50%, then they’re not “Later,” they are mere- 
ly a possibility. 


Remember, this isn’t a random bucket of 100 feature-re- 
quests. It’s a list of things that are important, possibly 
even seem urgent, that you really do want to do, that 
maybe only barely didn’t make the cut. In short, these 


were the difficult decisions. 


I’m actually as proud of the 
things we haven’t done as 
the things I have done. 
Innovation is saying no to 
1000 things.” 

—Steve Jobs. 


Full Example: Tesla 


The final result is clear and simple, hiding the complexity 


of the process that created it. 


but go ahead I guess.” 


Let’s use Tesla as the example, because although it’s over- 
used, that means you’re probably familiar with it, and 


thus you can see how it all comes together: 


Strategic Objectives 
Tesla’s long-term vision is to accelerate the world’s transi- 
tion to sustainable energy through the widespread adop- 


tion of electric vehicles (EVs). 
To do that, we need to: 


1. Invent vehicles that are not only battery-powered but 
also desirable, because of their performance and 
safety. 

2. Create production lines that are scalable but also 
cost-effective, so we can be profitable. 

3. Generate excitement and loyalty by bucking 
consumer expectations that EVs are impractical and 


have low performance. 


Obstacles 


These obstacles stand in our way of achieving our vision: 


¢ Intrinsic difficulty: Inventing new types of battery 
and production lines is near-impossible. 
e Market Skepticism: Overcoming consumer and 


industry skepticism about the viability of EVs. 
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¢ Manufacturing challenges: Scaling up production to e Safety and Practicality: Although we believe 
meet demand. eventually we will make the safest sedans in the 

e Infrastructure: Lack of widespread EV charging world, first we have to solve the challenges of 
infrastructure creates range-anxiety. batteries and performance, because those are the 


obstacles. We know that safety is a matter of good 


Main Activities design, but we already know from existing EVs that if 


We will overcome these obstacles while achieving our strate- the car is safe, but not performant and low-range 
gic objectives by: then it will not be popular. So we have to solve those 
first. 


¢ Innovate in Battery Technology: The insight is that 


e Affordabl dels: Th ill trul 1 
the battery is the key. Develop more efficient, longer- Nts re ce ag eee ee gee 


. . . and make the impact we want in the world, when it 
lasting, fast-discharge batteries. 


; 2 is accessible to most people. But at first, before 
¢ Direct-to-Consumer Sales: Bypass traditional Peop 


. _ factories are at scale, before the bugs are worked out, 
dealership models to control pricing and customer 


. before all the costs are minimized, our cars have to 
experience. . 
be expensive. So we have to start out as a luxury 
brand. 


¢ Expand to China: The second-largest market in the 


e Scale Production: Invest in Gigafactories to 
massively increase production capacity, hiring 


specialists and investing with the insight that 
: : world, eventually will be the largest, and the first 
factories are not just a necessary step but 
: . manufacturer there might have a decisive advantage. 
strategically vital. 


. But it is its own special challenge, and we need a 
e Infrastructure Development: Invest in a network of 


: . . popular car that works before we jump into that 
Supercharger stations for convenient charging. 1 
pool. 


Metrics & Indicators 


We know whether these activities are working through: 


oe . Tesla is often lauded for having a clear strategy, and 
¢ Battery Cost and Efficiency: Measure improvements 


. ; ; . ou’ve heard them report the metrics above as indicators 
in battery cost per kWh and immediate-discharge 2 P 


of progress or suCcCeess. 
rate. 


e Vehicle Sales and Market Share: Track the number You need a plan that is just as clear Each team needs 


of units sold and market share in the EV sector. one, and your company needs one. 


e Production Capacity: Measure fully-completed 


cars/day per Gigafactory. Otherwise, you're probably not working on the right 


Infrastructure Expansion: Number of new things, and certainly not communicating what must be 
Supercharger stations operational. done, in a way that everyone can understand and 
¢ Consumer Delight: People absolutely love their implement. 


Teslas, calling it a magical experience and never 
: 8 8 *P . Tell your story. 


wanting a gas-powered car again. (not a number) 


Not doing Printed from: A Smart Bear 


We are intentionally not doing these valuable, important https://longform.asmartbear.com/strategic-planning/ 


things, so that we can focus on the above: Te 
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How many things should there be? (Hint: Not 10) 


by Jason Cohen on December 10, 2023 


If we happened to evolve with nine fingers, we would have 
“Top 9” lists. So, a “Top 10” list probably doesn’t have the 
correct number of things. 


How many things should there be on the list? 


17: One per team 
If you have 17 teams, perhaps you have 17 items. 
Normally this is too many items, but this is actually a 


repeat of “1: Focus” at the team level. It’s still arguably 


R.1.S.. 
E.F.F.0.R.T. 
A.T.T.A.1.N. 
C.L.O.S.E. 
H.U.S.T.L.E. 


MM 


“Now I know that's a lot of acronyms, so here’s 
an acrostic to help you remember...” 


1: Focus 
It’s reductive to say only one thing is important, but it 


also ensures that it gets done, as quickly as possible. 


3: Magic Number 

Humans across the globe have always liked when 
things come in threes. Also you can remember them 
all. 


7: Because 7 + 2 


en items is what the average person can keep in their 
head simultaneously. If you can’t keep it in your head, 
even when you're trying very hard, it’s too many 


items. 


5: Because with post-COVID brains, now it’s 5 + 2 
I have no data to support this claim, but I claim it 


nonetheless. 


too many if people span multiple teams. 


132: 100% completion 

If there are 132 flags in the security audit, you could 
set a goal to address all of them. Getting through 
100% of a specific set of things is a powerful state- 
ment. One could call this the same as “1: Focus” but 
with sub-items, where it’s nice to have a checklist, and 
nice to feel progress as the team gets through the 
checklist, and clarity on why there are 132 items 
specifically, and a clear state-change when the very 
last item is finished (e.g. “We pass this audit, and then 


[large customer] renews for three years.”) 


402,302: Bankruptcy 

An inbox with this many unread emails is the same as 
an inbox with no unread emails, except a number 
looms over your head. Declare “bankruptcy”—whether 
it’s the email inbox or the Jira backlog. Archive? it all. 


The important things will come back. 


1 “archive,” rather than “delete,” because disk space is close to 
free, and if you really needed something after all, you still have 
it. Not quite Marie Condo. 


Always Ask: What is the purpose of this list? 


Given its purpose, should there be more things or fewer 
things? Is there a good reason why this the right number 


of things? 
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How many things should there be? (Hint: Not 10) 


Usually, fewer is better, and the right answer is almost never “10.” 
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The roadmap to Product/Market Fit... maybe 


by Jason Cohen on December 24, 2023 


This eight-step process brought WP Engine from an idea to 
a Unicorn. While there are other roads to Product/Market Fit, 
consider copying some of these ideas. 


! 
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“What was initially thought to be a simple process is 
in fact an incredibly complicated, intricate, and complex 
system that I've codified and organized into a few 
easy-to-follow rules that are more difficult 
to implement than you’d think.” 


What is the formula for going from an initial product 
idea to Product/Market Fit—a company that is growing 
and sustainable, with customers that want to pay and 


want to stay. 


(Assuming you even agree with that definition of 


Product/Market Fit.) 


There isn’t a fail-safe roadmap of course, but there is a 
progression that describes how WP Engine became a uni- 


corn, supported by 17 years of articles from this site. 


Here’s what that progression looks like. Then we'll ex- 
plore the other truth—that it often doesn’t work like this 


after all. 


1. Personal Fit: “Passion” is useful, but winning 
requires a personal edge. 

2. Market Fit: Most good ideas aren’t good businesses. 

3. Customer Fit: Talk to customers before you waste 


months building the wrong thing. 


4. Build and ship the SLC quickly: It’s where the real 
learning happens. 

5. Marketing and Sales more than writing code and 
tweaking design. 

6. Retention-driven Product development; attend to 
your existing customers first. 

7. Prioritize systematically, ruthlessly, strategically. 

8. Manage your psychology on a journey of self- 


discovery and constant rejection. 


1. Personal Fit: The inner fire, leveraging 
a personal edge to do what you were 
meant to do 


A great idea or a great strategy that you can’t execute 


well, isn’t a good strategy for you. 


You start with nothing—no product, no customers, no 
brand, no distribution, and compared to competitors, no 


money and no time. 


To succeed, you need something. Something special. 
Something that gives you an edge, despite being woefully 
inadequate in every dimension. Is “passion” that 


something? 


Passion 


Do what you love, and the money 
will follow.” 


—Motivational advisors 


That is false, as evidenced by most artists, philosophy 
majors, and the 80% of startups that fail despite 


founders’ genuine love and obsession. 
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Passion is required; it’s just not enough. Passion is the 
motivator, especially in the dark times when your savings 
is depleted, competitors are beating you, as you suffer a 
constant bombardment of rejection from potential cus- 
tomers and potential employees and potential investors, 
as you question your self-worth and wonder whether the 
cause of your Impostor Syndrome is that you’re actually 
an impostor. 


You have to be burning with an 

idea, or a problem, or a wrong 

that you want to right. If you’re 

not passionate enough from the 

start, you'll never stick it out.” 
—Steve Jobs 


You need to articulate exactly what your passion is, so 
that you can use it as a filter for business concept. This is 
difficult; some people spend their whole life trying to fig- 
ure it out. Jerry Colonna says “The purpose of life is to 
discover who you are, and then live fully into that.” It is 
not an exaggeration to say that a startup is a reflection of 
who you are; that’s why they call it “your baby.” 


See the last section of this article on personal fulfillment 
for a list of questions that help you suss what your pur- 


pose and passions are. 


Your goal is to find the work you were meant to do. 
Your calling. The work that you would do for free, but 


more—what you are compelled to do. 


‘Listen, I’m fine with the robe, the sorcery, even mixing 
potions in the bathtub. But for god’s sake, Gary, 
would you please use a coaster?!” 


Still, the existence of passion doesn’t imply the existence 
of a business model. (We'll solve that in the next step.) 
Passion doesn’t give you a competitive edge, because all 
the other founders have passion, and larger companies 
have mountains of advantages that make “passion” look 
like a sling-shot attacking an aircraft carrier. You need an 


edge. 


Leverage 

“Leverage” means yielding a huge output from a given in- 
put—accomplishing an inordinate amount of quantity or 
quality, with relatively little time and money. You gener- 
ate leverage from the mixture of talents, taste, and expe- 
riences that you possess. To figure out how this works for 


leverage. 


Leverage is good, but unique leverage is far better, be- 
cause that’s your source of differentiation! from competi- 
tors and alternatives. Often this appears at the intersec- 
tion of your peculiar above-average talents and experi- 
ences, which taken together are unique” This uniqueness 
must then be coupled with a particular way that you are 
approaching this problem and solution, such that the 
combination of your leverage against your path is 
uniquely excellent, even for a small number of potential 


customers. 
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This leverage + path combination also forms the kernel 
of your strategy° as your competitive positioning. You 
need something that competitors lack, because your cus- 
tomers are going to compare you to competitors whether 
you like it or not, and you have to have an answer be- 
sides “we have this one minor feature that they don’t 


have” or “we’re $10 cheaper.” 


1 “Different,” not “better.” Yes, you will be “better” for according 
to some set of people, but definitionally that means you’re 
“worse” for another set. The latter might even be orders of 
magnitude more numerous than the former. That’s OK, and 
that’s why I don’t like using the word “better.” You should be 
distinctive, aligned with your calling and your strengths, so 
that the set of people who do find your brand of different to be 
“better” will flock to you, buy from you, and even love paying 
you for it. 


iS) 


The quintessential exponent of this idea is Scott Adams, writer 
of the Dilbert cartoon, who points out that he is a decent illus- 
trator but not a great one, a decent humorist but worse than 
any comedian on Netflix, and has held jobs in the tech sector, 
unlike almost any artist or comedian. It is in the intersection of 
being in the top 25% in each of these three “circles of compe- 
tence” that he is unique, not because he is top 1% or “best in 
the world” at any one thing. 


3 Leveraging strengths into durable competitive advantage is one 
of the six characteristics of great strategy. 


The key question that summarizes this intersection of 


“you” and “path” is: 


Why are you the perfect person to build this 


company? 


Be yourself. Everyone else is 
taken.” 
—Oscar Wilde 


2. Market Fit: A working theory of why 
they will buy. 


Once you inventory the inner world—the shapes of spa- 
ces where you were meant to play—you turn your gaze 
externally to figure out whether your idea works out in 


the market, in the world of customers, competitors, 


trends, problems-to-solve, jobs-to-be-done, and products. 
These things you do not control, and thus must under- 


stand, conform to, but also exploit. 


It’s right there in the name: “Product/Market Fit” means 
fitting into the market, not just building something that 
would be fun/interesting/edifying/curious/exciting/ego- 
enhancing. Those are all good reasons to have a side- 
project, but none is a reason why that side-project will 
become a profitable business, even at a scale that feeds a 
single person. That’s why most “indie hacker” startups 
and “AI is cool” startups fail—the genesis was “fun 


project,” not “plausible business.” 
“I had the problem myself, so I built a product to solve it.” 


This might be the most common origin story, tacitly con- 
cluding that “this must be a business because I would 
have been a customer.” Indeed, my startups all started 
this way. But your understanding of “the problem” as it 
pertains to you alone is much less likely to be a real busi- 
ness than you think. You are, in fact, not like your 


customer* 


4 For a start, your customer isn’t quitting their day job to create a 
business. 


Sometimes a passion project turns into a business any- 
way. That’s what happened to me at Smart Bear. More on 
that later. But that’s luck. The point of a framework like 


this one is to reduce your reliance on luck. 


“The way to get startup ideas is not to try to think of 
startup ideas. It’s to look for problems, preferably prob- 


lems you have yourself. 


The very best startup ideas tend to have three things in 
common: they’re something the founders themselves 
want, that they themselves can build, and that few oth- 
ers realize are worth doing. Microsoft, Apple, Yahoo, 
Google, and Facebook all began this way.” 

—Paul Graham, How to Get Startup Ideas 
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A theory of why they will buy 

You need a plausible theory of the customer, market, and 
business model. “Plausible,” because many ideas fall 
apart under honest scrutiny. “Theory” because of course 
it won’t be entirely correct, but it will be your working 
understanding of the world, which you modify as you 


learn. 


Here’s exactly how to develop a specific theory of the 


these market characteristics: 


1. Plausible: Do 10M people or 100k companies have 
the problem? 

2. Self-Aware: Do they know & care they have the 
problem? 

3. Lucrative: Do they have substantial budget to solve 
this problem? 

4. Liquid: Are they willing and able to buy right now? 

5. Eager: Do they want to buy from you, specifically? 

6. Enduring: Will they still be paying(-it-forward) a 


year from now? 


(and savings and heartache) building [insert name of 
failed startup]. Don’t be one of those people. 


You might think it infeasible to answer these questions, 
because the research is impossible or the idea is so new 
that existing data and trends are irrelevant. However, as 
you'll see in the article, you can do it using Fermi 
Estimation—a technique useful not only in market analy- 


sis but for ROI, probabilities, and decisions. 


BUSINESS PLAN 


HOPE FOR 
THE BEST 


“Well, it’s not the worst I’ve seen.” 


Early strategy 

Even better than having a great business model is to have 
a great strategy. Not a twenty-page document, but a one- 
pager that conforms to the guidelines in that article, ex- 
plaining “how we will win.” It’s never too early to be ask- 


ing yourself how to leverage your strengths to build 


products that are great and differentiated despite your 
weaknesses. It’s never too early to write down your as- 
sumptions and decisions, to ensure they are at least self- 
consistent if not self-reinforcing, so that when contrary 
evidence appears, you can notice that, and react 
methodically. 


To generate tangible strategic ideas for your business 
model or strategy—explaining “how we will win’—con- 
sider creating a theory of where your product will fit in 
the customer’s Needs Stack, pick a few of these tactics for 
navigating the fact that the future is unpredictable, and 
decide how you will generate “Love” and “Utility” types 
of Willingness-to-Pay. All of these are fun, accessible, 


strategic, and they work in practice. 


“But often the first idea is wrong, so why do I need a specif- 


ic idea with a specific business model?” 


Because walking in a random direction is not progress, 
and because the main way to discover the right direction, 
is to have a clear theory of the world, and notice when 
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the world contradicts it, so you can pivot into a better 
theory, and thus iterate into a genuinely great strategy 


and market-understanding. 


That’s what actually happened with Smart Bear, and 
Slack and WhatsApp and Flickr, as described_and_ana- 
lyzed here. Only by having a specific, strong, clear idea, 
was it possible to notice what customers really wanted in- 
stead, which led each of those companies to pivot into 


the idea that became successful. 


In all those cases, the idea was very personal to the 
founders—the personal-fit of passion for the solution and 
leveraging an edge to create something that resonated 
with early users. But it was only with a specific theory of 
business model and strategy, which was then negated by 
reality, and then an intentional pivot, to achieve 
Product/Market Fit. 


3. Customer Fit: Find the ideal customers 
first. 


There’s only one source of truth for what customers will 


buy: Customers. 


Not advisors, not experts, not analysts, not Twitter polls, 
not research, not past data, not even competitors’ behav- 
ior. You have to talk to customers. 


\ aeideascnl 


“So, tell me a little bit of what you think I want to hear 
about yourself.” 


Customers are fickle. You ask whether they'll pay $100 if 
your product does 
build it and they don’t buy. So why talk to them at all? 


and they say “yes;” then you 


Don’t you need to put the actual product in front of 


them, and see what they actually do? 


Customers can tell you what their lives are like, which is 
how you validate your business model and strategy from 
the previous section. Customers can tell you what they 
won’t buy, which has happened to me repeatedly. You can 
discover that the average customer doesn’t know the 
problem exists, or doesn’t have a budget for it, or isn’t 
prioritizing it. When they say “yes,” it’s a “maybe,” but 
when they say “no,” it’s a “no,” and you just saved your- 


self months or years of wasted time. 


You can find out where they go to find products like this, 
so that you can advertise in the right place. You can find 
out what language they use to talk about the problem or 
solution, so you can copy that language in your advertis- 
ing and social media home page and capture their atten- 
tion. You can find out how their budgets work, so you 
can price and package and position accordingly. You can 
find out how they’re addressing the problem today, so 
you know what you're selling against, whether that’s a 
competitor, an alternative, or something they're doing by 
hand. You can find out what causes them to break out of 
their daily life and say as Bob Moesta puts it, “Today is 
the day ’m going to buy __,” so that you can try to 
be there when that event happens, or possibly even cause 


it. 


It is easy to find examples of successful companies who 
never asked customers what they wanted. That happened 
to me at Smart Bear. But again that relies on luck, and 
most of the time, you don’t get lucky. This is not a good 
way to gamble the next few years of your life. 


But you don’t want to do this work. You want to build 
the product because that’s the fun part. Potential cus- 
tomers are hard to find and they don’t want to talk to 
you. It’s going to be like that when you have a product 
too, so if you can’t do it now, you won’t be able to do it 
later. Building the product first won’t make it easier to 


find or talk to customers. Building the product first will, 
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however, ensure that you haven’t actually built what peo- 
ple want, because you never found out what people 


want. 


Justin @ 

@JustinQuda 
It’s taken years for me to change my mindset from “I know best” to 
“what does the user want?” 
I've been building a startup in my own echo chamber. 


Today | did an 1 hr call with a Quda creator - it was awesome! 


Talk to users - they won’t bite. They want you to succeed too. © 


How to interview customers 

Once you get someone on the phone, how do you inter- 
view them in a way that maximizes learning and leads to 
a specific theory that you can execute on, from your 


unique winning advantage to pricing? 


The customer validation system I’ve developed is the 
Iterative-Hypothesis customer development method. 
This provides you with goals that you achieve using in- 
terviews, how to create the hypotheses that will drive 
your business model, and how to write questions that 
maximize learning. 15 years ago it invalidated a startup 
idea that I thought was good, and then validated the 
startup idea that became WP Engine—now a unicorn. 


How to get customers to talk to you 

The way I got interviews for WP Engine is by using 
LinkedIn to find people who had the title and industry I 
was targeting (web developers in WordPress), and asked 
them for an hour of their time to chat about a new start- 
up concept for whom they are the ideal customer. 
Furthermore I offered to pay any amount they wished for 
that hour, because I value their expertise and their time; 
I’m not asking for a donation, I’m genuinely interested in 
their expertise. Out of 50 requests, 40 agreed, and only 


one asked to be paid. (30 eventually became customers.) 


Another technique is to buy ads for a landing page with 
an email sign-up form and web-chat, and talk to the peo- 


ple who arrive. Ask what they're looking for, what they 


need it for, why are they looking for it right now, how 
they found you, and what else theyre considering. When 
it goes well, ask them to hop on the phone and run the 
full method above. 


Another technique: Go to an event where your target cus- 
tomers are, walk the halls, and interview everyone all 
day long. People like talking about themselves, and you 


get to hone your pitch with 100 practice sessions. 


Here’s more: 


Christopher Gimmer @ r Fotiow 68 
g @cgimmer 
About a month ago, | had an idea for a new SaaS app based on the pain | 


was experiencing with the market leader that completely overhauled 
their product for the worse. 


Here's a few things | did to gain more confidence in moving forward: 


1) Reached out to 10+ people in my network to see if they had paint 
points with [X] and did customer research interviews. 


2) Sent DMs to random people on Twitter who were complaining about 
[x]. | told them we were thinking of building a competitor and if they'd be 
willing to hop on a call to share their current pain points. Some replied 
and agreed to hop on a call. 


3) Read a bunch of G2 reviews of the incumbent to see if peoples' pain 
points matched mine. 


4) Dug through many Reddit threads to see what the common pain 
points were. 


By the end of all this, | was pretty confident that many other people were 
experiencing the same pain points as me and that there was a 
willingness to pay. 


| then analyzed all the available marketing channels and put a plan 
together on how we'd get traction. 


Going to be announcing the new product soon #@ 


Find your ICP 

Your goal is not only to validate your theory of the mar- 
ket, but to discover your ICP (Ideal Customer Profile). 
This is your “perfect customer”—a segment almost comi- 
cally over-specified to be so perfect for your product that 
you are truly the best choice in the market, and they 
would be crazy not to buy. You will then aim all of your 
marketing messaging at this person: Website, advertise- 


ment, terminology, writing style, pricing. When the ICP 
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lands on the home page, it should be obvious in three 
seconds that this is perfect for them, that you “get” them, 
and they will be pulled through the process until they're 


a customer. 


GA Bryan Smith @ 

‘Ay = @OrionSeven 

The biggest mistakes | made in my first startup was planning an MVP far 
too large. 


The root cause for that was | didn’t figure out my ideal customer profile. 


Instead of building for everyone | should have built for someone specific 
which would have let me build far less. 


You are scared that targeting only the ICP limits your po- 
tential market, but this is not what happens. In “Selling 
to Carol,” I explain the mechanism and provide examples 
showing that for every ICP there are 10x more people 
who make similar buying decisions, and 100x more who 
take more convincing but ultimately also agree, and 
therefore you end up selling to a far wider market than 
you feared, yet earning attention and loyalty from having 
a clear, unique message. Your company, brand, and prod- 


uct will actually mean something. 


“I’m looking for something in a small pond.” 


Find and talk to the customers; they’re the only ones 


with the answers. 


4. Build and ship the SLC. 


Interviews give you a better model of the world, but the 
real learning doesn’t start until people are using—or 


more often, not using—a product. 
SLC is my alternative to MVP: 


Simple, because complex things can’t be built quick- 


ly, and you need to ship product to customers now. 


Lovable, because crappy products are insulting to 
customers, and you didn’t start this company to 
make crappy products. The love overpowers the fact 
that it’s buggy and feature-poor. There are many 


of “Love.” Pick a few. 


Complete, because products are supposed to accom- 
plish a job. Customers want to use a v1 of something 


simple, not v0.1 of something broken. 


Build something small (but also lovable and complete), 
and ship it to customers, because that’s when real 


“progress” begins. That’s how you learn the truth. 


If youre building for six months and a customer hasn’t 
used it, you’re not on the road to Product/Market Fit; in- 
deed you’re not making progress on the business at all. 
What you’ve done is proved what you already knew— 
you can build software. 


The learning starts only when customers are (trying to) 


use it. 


S Brandon # Flightcontrol @& 


@flybayer 
Yo, watching @LogRocket recordings of people using your new app is like 
the most painful thing in the world 4 @ 
But wow, | cannot imagine living without it. People just won't tell about 


all the little issues and confusions they have 


5:42 PM - Nov 16, 2023 - 6,449 Views 
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5. Focus more on marketing and sales, 
less on product. 


Your natural inclination is to do what you enjoy and un- 
derstand, so you need to press yourself to do so much 
marketing and sales that it feels like “too much.” 


Ask yourself: What can you do today, that will (a) get 
more people from the target market to come to the web- 
site or (b) convert more of those people to try the prod- 


uct or (c) convert more of those to buy the product. 


The answer might be found in writing more code. 
Improving the new-user experience, for example, could 
increase the conversion rate from “try” to “buy.” But more 
often the answer isn’t inside the product; indeed several 
of those steps happen before the person has used the 
product at all. 


You'll advance the product anyway—I know that. And 
you should, but you don’t need to be told to do that. 
After all, it’s all you ever wanted to do, even at step one. 
That’s why you need to set an intention every day to 
make progress on one of those questions. That’s how 


you're going to generate growth. 


6. Prioritize retention-based product 
work. 


What does it mean, that one person out of billions cared 
enough about your problem-space to notice your adver- 
tisement or social media post, then felt compelled to fol- 
low the link, then was so intrigued by the website that 
they joined the 1-out-of-100 that tried the product, then 
liked what they saw enough to take out their wallet and 


pay, then started to use it... and then cancelled?!? 


After all that—clear signals of desire, need, willingness to 
pay, and apparent “fit’—they decided “Nope, this is not 
what I want.” 


What it means is: You made the right promise, but didn’t 


keep it. 


Marketing is where you discover the promise the cus- 


5 


tomer wants you to make, and retention” is where you 


discover whether youre fulfilling that promise. 


5 Actually, retention is a lagging, multi-factor indicator, which 
makes it useful as a high-level check, but not useful as an oper- 
ational day-to-day metric. Here’s a complete system for practi- 
cal metrics that embraces this fact and points to the right set of 
metrics. 


itr 
A. y, . 
So oF, 1 


— 


"No, you're thinking of Cupid. I'm the 
one-night-stand fairy." 


Some people believe that low retention is fine because 
you can make up for it with growth. That’s not true for 


two reasons: 


1. Growth will start out good, but then will slow and 
halt sooner than you realize® 

2. If customers don’t want to use the product, your 
“srowth” is fake news. It’s not Product/Market “Fit” if 
the “market” decides “not a fit” after actually using 


the product. 


6 See these articles with data from Product/Market fit companies 
and the math behind growth grinding to a halt at companies 
with low retention. 


Therefore, when you do work on product features, build 


things that increase retention. 


This starts by understanding why people cancel. Gaining 
this understanding is difficult, because once they leave 


they don’t want to talk to you anymore. You might coerce 
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them by paying for interviews with Amazon gift cards or 
charitable donations. Some people will respond to an 
email (no harm in asking). Better is to look for signs that 
the customer isn’t being successful (e.g. activity within 
the product) and reach out before it’s too late, while 
they’re still in the middle of their struggle, and therefore 


might be willing and able to articulate it. 


Remember that your goal is not to discover what unsuc- 
cessful customers have in common, because successful 
customers often have those things in common as well. 
Rather, it is to find patterns in unsuccessful customers 
that are not shared by successful customers. Those are the 


attributes that lead to action. 


For example, if “using feature X” means people are suc- 
cessful, maybe unsuccessful customers don’t know about 
it (So you promote its usage), or maybe they can’t use it 
because it’s incompatible with their workflow (so add in- 
tegrations or options). Sometimes, however, it’s about 
the customers themselves, where tuning the ICP means 
you find more customers who are already a fit, perhaps 
targeting different industries or geographies, different 
stages or sizes of company, the title or attitude of the 
buyer, or the specific use-cases they are trying to solve 


for. 


The Talk/Walk workshop might help, because it helps 
elucidate the difference between the promise and the ex- 
ecution. But probably, as usual, you just need to talk to 


customers. 


7. Ruthlessly, methodically, systematically 
prioritize. 

More than just “focus,” more than a generic admonition 
to “prioritize,” use a system that ensures you’re leverag- 
ing your most valuable, most limited asset (time) to max- 


imize progress. 


Do this by combining Fairytale Planning with the Rocks, 


Pebbles, Sand work-prioritization system: 


Identify the next milestone. 
At all times, be crystal clear about the single most im- 


portant thing to achieve. 


(Ex: Finding a good business model. Public launch. 20 
paying customers. Getting trial conversion rates from 
1% to 5%. Getting cancellation from 7%/mo to 4%/mo. 
Getting to $10k MRR so you can quit your day job.) 


You have to be executing against the next milestone 
every day. Write it down and look at it every morning. 
Everything else is a distraction to be ignored, or a nec- 
essary evil that you should dispatch as quickly as pos- 
sible, including delegation or accepting that it’s fine if 


it’s executed poorly or late. 


Identify the current obstacle to achieving the 
milestone. 
At all times, be crystal clear why it is difficult or slow 


or expensive to achieve that milestone. 


(Ex: Can’t figure out how to leverage your strengths for 
something differentiated and desirable. Easy to get 
someone to use it once, but only 15% use it again. Got 
17 customers from Product Hunt but now there’s no re- 
peatable way to get more. Can’t get people to agree to be 
interviewed. SLC isn’t accepted by beta testers. Freemium 


users don’t have enough incentive to convert to paid.) 


You must face the difficult truth. Do not pick the ob- 
stacle that feels comfortable; pick the existential crisis 
that is uncomfortable exactly because it is critical and 
scary and true. 


You have to be attacking or side-stepping the obstacle 
every day. Write it down and look at it every morning. 


Split work into Rocks, Pebbles, Sand. 
Use this specific Rocks, Pebbles, Sand framework to 


prioritize and schedule different types of work with 
the correct philosophy and process appropriate for 
each type. 
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Select the Rock that attacks the obstacle and mile- 8. Manage your psychology. 
stone together. 

Use Binstack to prioritize one Rock that addresses that 
most important milestone and attacks the obstacle. 


(You don’t have time for more than one.) 


If you can’t think of one that’s good enough, don’t just 
proceed with a mediocre plan that will occupy the 


next three months; grab some friends—or better yet, 


customers!—and brainstorm a better one. 


Schedule Pebbles sparingly. “So you'll never be part of a royal flush. You know what? 
Because you have to focus on the Rock, and you have Me neither. And that's OK.” 


little time for anything else, use this ROI framework to 


prioritize just a few other activities that are valuable 
ae ; = : Throughout this a step-by-step roadmap, you face de- 
enough to justify spending your remaining time. 
manding psychological challenges. 


Almost everything should be only “good enough” 


e =: . M 
You have taste, you have craft, and part of the reason Faciny thes teuth for alowing Wy Hauyeuwish Were 
; Sa true” to get in the way of finding out what is true, 

you're doing all this is to express yourself. But almost 

; , ‘ about your abilities, customer’s wishes, the shape of 

everything you do will not determine whether the 

the market : 

company is successful. (Repeat that last sentence until swhy people leave 
: pare ; : ¢ Finding yourself—coming face-to-face with who you 

you fully internalize it.) Which means most things 


d wh t, figuri t what i 
should only be “good enough.” Let it go. eae Ae a ey pe, ee ee 


actually important versus what is necessary or 


Most metrics should be satisfied, not maximized. Pick temporary, enjoying the journey especially when 
KPIs accordingly. even a successful end result is often not what you 


hoped it would be. 


Having said all that, sometimes the very best aspects : ene oat : 
8 2 vy P e¢ Making clear decisions—deciding what youre not 


of a design are the little things, the obsessive things, shod aivanditow trying ta dort anyway cdecidiny 


the things you are compelled to make just so. Whether jolt placed themarkel: and noreyine te beall 


web design or UX design or a “perfect” feature which things to all people; deciding your ICP and having 


is only perfect because of the myriad corner cases that : : bat 
yP yr the fortitude to go all-in; deciding the current 


took a surprising amount of code and care to cover, . : ; ; 
P 8 ° milestone, the main challenge, the one big thing you 


but which means a new customer has an amazing ex- nae es . 
have to do, and sticking to it without getting 


perience. Pick your battles. Aiea. 

Don’t stray from the system. You don’t have the time. * Constant rejection—your ideas won't be good 
enough, your customer interviews will negate key 
assumptions, 99% of website visitors won’t take an 
action, long sales calls will result in nothing, 
customers will buy but then leave, employees won't 


join you, investors won’t invest. 
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¢ Fluid change—despite clear decisions, rejections 
often demand a change, which is difficult to accept, 
difficult to enact, difficult to admit when things are 
wrong, and difficult to decide whether some 
rejection should be ignored as an anomaly, or 
whether it’s indicative of a “learning” that must result 
in a change. 

¢ Acting while uncertain—being “all-in” even while 
knowing you must be wrong about some things, even 
while getting rejected and making changes, even 
while doing things you’ve never done while experts 
tell you what to do differently, even while feeling like 
an impostor who has no right to be here, even while 
knowing you have no idea what you are doing while 
it appears everyone else knows exactly what they’re 


doing. (Don’t worry, they really don’t.) 


This might be the hardest thing you’ve ever done. It’s a 
gauntlet of pain, even if it’s also the most exciting thing 
you'll do in your professional life. Many people stop be- 


cause it’s so hard. You can’t blame them. 


But you’re not going to do this, are you... 


You have a fun feature idea so you’re just building it. 


You always wanted to learn Rust so you’re just learning 
it. 


AI is astounding and it’s the future so you’re building 
things with it, even though it’s a technology and not a 


customer’s problem that needs solving. 


You’ve started your own company in part because you 
don’t want to be shackled by “processes” and “frame- 


works,” so you’re just doing whatever you want. 


Visual Studio is comfortable and Google Ad Manager is 
foreign and like Las Vegas is designed to separate you 
from your money, to say nothing of the discomfort of ask- 
ing for the sale or getting someone on a call to talk about 
their workflows or get an earful about how your precious 


software actually sucks. 


Uncomfortable. Scary. And you’re lost, unskilled at those 
things, not making progress, not even wanting to. So you 
slip back to Visual Studio where you know what to do, 


and enjoy doing it, and you’re good at it. 


Then you post on Twitter about how you have $1600 in 
MRR after 18 months but it’s okay because perseverance 
is a virtue. When in fact you’ve fallen into the well- 
known trap of doing what you’re good at, what you love, 


but not what the business needs done. 


Perseverance is a virtue, if you’re doing the right work, 
with the right goals. It’s a vice if it means you’re moving 


diligently in the wrong direction. 


Smart Bear only halfway followed the 
roadmap 


But who am I to talk? 


The company I founded in 2002, sold in 2007, and that 
gave me the online handle that persists to this day on 
this site, Smart Bear’s first “product” Code Historian was 
a personal project, built because I want to learn .NET 
(we would later rewrite everything in Java), built be- 
cause I thought there was value in the historical record 
encoded in version control (all products that operated 
under that assumption failed). 
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i [Code Historian] C:\proj\httpd-2.0\server\util_md5.c 
File Options Help 
Sariieem|Recent Changes y| in C:\proj\httpd-2.0\server\ util md5.c 


HB 11 months 17 months 23 months 23 months 35 months 


#include “ap_config.h” 
#include “apr_porteble.h" 
#include “apr_strings.h” 
#include “httpd.h” 
#include “util_md5.h” 
#@include "util_ebedic.h” 


JAP_DECLARE(char *) ap_md5_binary(apr_pool_t *p, const unsigned char *bu 
6} { 


const char *hex = "0123456789abcdet"; 
apr_mdS_ctx_t my_md5; 
unsigned char hash[MD5_DIGESTSIZE]; 


char *r, result(33]; /* (MD5 DIGESTSIZE * 2) +1 */ 
int i; 


yt 
* Take the MDS hash of the string argument. 
"7 


7 apr_mdS_init («my_md5); 
#if APR_CHARSET_EBCDIC 
apr_mdS_set_xlate(smy_md5, ap_hdrs_to_ascii); 
gendif 
apr_mdS_update(smy_ md5, buf, (unsigned int) length); 
apr_mdS final(hash, «my_md5); 


for (i = 0, r = result; i < MDS _DIGESTSIZE; i++) { 
*r++ = hex[hash[i] >> 4]; 
*r++ = hex[hash[i] « OxF]; 


wuthor: dougm, 
prefix lisapr functions 
and types with a 


ee 


This doesn’t make me look old, right? Right??? 


I didn’t talk to customers ahead of time. I did talk to 
them a lot after they started paying, and learned that 
many didn’t really want Code Historian at all, but rather 
they were abusing it to accomplish something else (in a 
story told here). As a result, I created Code Reviewer, 
which sort of hit the need in question, but the architec- 
ture was all wrong and the features were all hacks. But 
all this customer feedback gave me the confidence to re- 
write everything from scratch (which everyone says you 
should never do), resulting in a product called Code 


Collaborator, which made millions of dollars. 


Would I have found the idea for Code Collaborator by in- 
terviewing customers using the “Iterative-Hypothesis” 
method above? No, because I wouldn’t have known what 
to ask them, and they wouldn’t have known to tell me. I 
would have discovered that Code Historian wasn’t a good 
business idea... and that’s it. Maybe I would have had a 
completely different and better idea. Maybe. But the fact 
is, the actual “path to success” was to create something I 
liked, just interesting enough to attract initial customers 
even though it’s not a great business, and then pivot to 
Code Reviewer, and then pivot to Code Collaborator. The 
interviews would have stopped everything cold. Does 
that make “interviewing” the wrong. road _ to 
Product/Market Fit? 


I do think building and getting product into customers 
hands is far more valuable than conversations. Smart 
Bear was an example of (1) having a specific idea match- 
ing my passions, which then because of (4) building and 
shipping and (5) marketing and selling and (6) building 
for retention by talking to customers, lead to the right 


idea. So, it was part of the path, but not the full path. 


So, while the path to success didn’t honor every step in 
the roadmap, it still would have failed if I had violated 
those other steps—coding in isolation, coding instead of 
selling, coding instead of honestly figuring out what cus- 


tomers really wanted. 


WP Engine followed the roadmap and 
became a unicorn 


That said, with WP Engine I traveled this roadmap 


exactly. 


WP Engine wasn’t the initial idea. The first idea was for a 
marketing analytics tool with features that Google 
Analytics lacked: A real-time event dashboard, a clever 
goal-setting-and-measuring system that worked retroac- 
tively, and the ability to capture web-form data even 
when it was only partially filled-in and never submitted, 
all with just one line of Javascript’, But I ran that 
Iterative-Hypothesis method and after 20 interviews I 
learned that the idea wasn’t strong enough® I was on the 
road; I (1) had a specific idea that (2) was clearly a great 
market and (3) vetted it with customers. Thanks to (3), I 


did not waste years on that project. 


7 These features are typical of analytics tools today, but in 2009 
it was innovative. 


8 This article gives the details of this invalidation, and how it 
was different from WP Engine, which the same method 
validated. 


I then had the idea of a managed WordPress platform be- 
cause I needed it myself—my blog would crash when I 
got to the top of Hackernews (a link-sharing site). I asked 
other bloggers what they use to keep WordPress fast and 
scalable; many said, “I don’t know, but if you find it, tell 
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me, because I need that too.” I ran the Iterative- 
Hypothesis method again, and this time it was very clear 
what the need was, what people would pay, and how to 
find them. And the market was already huge (WordPress 
powered 11% of the largest 10 million websites) yet 
growing fast (in 2023 it’s 43%). I knew WordPress be- 
cause I was a user, and for nearly twenty years I’ve loved 
code optimization problems, so this was a problem I un- 
derstood and am ideal to solve, where customers agree 
the problem exists, are already spending money on it, 
and the market is large and growing. That’s steps (1), 
(2), and (3), but this time the concept was validated. 


I built just enough software to fulfill the promises of 
speed and scale, building the first version in 30 hours. 
Good enough to not only stay alive and fast on a 
Hackernews day, but with almost no load on the server, 


which meant I was still far away from a capacity limit. 


Connections YW regis 


Bibel 


2010-05-02 12:00 2010-05-02 20:00 2010-05-03 04:00 24 2010-95-03 12:20 2010-05-03 20:00 


Hundreds of simultaneous connections on a Hackernews day 
in 2010 


The first version didn’t even have a customer dashboard. 
I took credit cards with the default web-form supplied by 
the online credit card service, and the rest was just 
WordPress. 


The first version of the website was... “spare,” shall we 
say? 


WPRENECING 


Better, Faster, Stronger. WordPress. 


We're developing a rock-solid, fully managed, hosting platform 
for Wordpress. We're designing a beautiful dashboard to control 
backups, staging, and plugins. Enter your email below to learn 
more about our beta. 


Submit 


The second version of the website was so bad, there were 
database errors shooting out the bottom. Database er- 
rors, on the home page, of the company that you’re sup- 


posed to trust with your precious website. 


Sign Up 


Feel free to read our ; policies. 


menu_order, post_title ASC 


Home 
WordPress database error: [No database selected] 
SELECT * FROM wp posts WHERE (post_type = ‘page’ AND post_status = ‘p 
In = 77 oF Th = 11? \) ORNFR RY menu order. post title ASC 


The market need must be overwhelming, if customers walk 
through this wall of fire and say “yeah just take my money.” 


So that’s step (4), the SLC product that fulfills the job 
completely with the absolute minimum of “Product.” 


We reached $1M ARR after 18 months. The reason—be- 
sides having the right product with the right promise in 
the right market, which was elements (2) and (3)—is 


step (5) focus on marketing and sales. 


Because I did marketing and sales before even having a 
product login screen, I was able to launch with 30 paying 
customers. Sure I worked on the product and worked 
support tickets, but what I focused on was how to get 


customers: 
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QUESTION FOR YOU: Are there places we should be advertising? Sponsoring? Pushing our affiliate program? 


QUESTION FOR YOU: Do you know other blogs (on any platform) that should be using us? Please make an introduction! 


From an “advisory group” email update from 2010; my asks 
were for marketing and sales, not product and engineering. 


When you do marketing and sales, with a product that 
people actually need, in a large and growing market (all 
covered by the exercise under (2)), then you might have 


a nice growth curve: 


—— ARR —— NAccounts 
1.2M 700 
IM 600 
500 
800K 
2 
2 400 § 
3 
= 600K 8 
300 xX 
2 
400K 
200 
200K 100 
0 0 
oo we oo os os os oo oo os os 
ws yy cer WOT yo na ened yr ce? wor 


Hit $1M ARR in first 18 months at WP Engine. 
But wait, the Product/Market Fit is still to come... 


We’ve always spent a lot of time on (6) retention. 
Sometimes years would pass where we wouldn’t add a 
new feature, because all our engineering time was spent 
on the challenges of scale and continuing to improve on 
the promise of speed and uptime. As a result, we’ve had 


best-in-industry customer retention for 14 years. 


High retention due to happy customers folds back into 
growth through word-of-mouth referrals and a thousand 


top-of-class reviews: 


WP Engine 


e Reviews 1,007 ¢ Excellent 


ESESESESES << 


VERIFIED COMPANY 


wPengine 


Reviews * 4.8 


1,007 total 

5st 87% 
C1) 4-star 2% 
{__] 3-star 1% 
C) 2-star 2% 
CJ 1-star 8% 


All that, combined with the hard work, the craftsman- 
ship, the large and growing market—did I mention the 
large and growing market?—and yes also the luck, 
meant that in early 2012 we hit Product/Market Fit, as 
further detailed here: 


2010 2011 2011 2012 2012 2013 2013 2014 2014 


The previous chart’s “up and to the right” growth from 2010- 
2012 looks almost flat compared to the sudden moment of 
Product/Market Fit, when it became hyper-growth, and never 
slowed. 


This roadmap worked for WP Engine, and it worked for 
others. Consider this thread from Simon Hoiberg, ex- 
plaining why his second bootstrapped company succeed- 
ed (FeedHive) where his first failed. This roadmap echos 
within it—playing in a large, growing market, finding 
and focusing on an ICP building only an SLC to start, and 
treating marketing and sales as more important than fea- 


tures and innovation: 
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a simonhoiberg 


My first startup never made any money. 


My second startup passed $10K MRR in year 1, and is doing +$1M ARR today, 3 
years later. 


What did | do differently? 
Let me share the lessons learned “ 


eaev 


a simonhoiberg 
> | talked to users and listened instead of telling myself that | knew better. 


> | built in a very well-established market with verified demand instead of 
trying to innovate. 


> | consistently marketed my product - slow, but sustainably - instead of 
trying to "blow up" using the latest growth hacks. 


VQea V 
@: 


a simonhoiberg . 
> | kept my product slim but effective instead of trying to solve everything 


for everyone. No one likes a "feature creature". 


> | put UI/UX on a pedestal and had a huge focus on *how* we solve a 
problem, instead of just solving it "somehow". 


~> | focused on users who truly appreciated the unique way we're solving 
problems differently instead of trying to target everyone. 


9Ae8 Vv 


Do you have to walk this road like Simon discovered with 
FeedHive and I discovered at WP Engine, or can you 
stumble onto it halfway through like Smart Bear, or can 
you take a completely different route and just emerge at 
the finish line? Or can you just cherry-pick the things you 


like from this framework, and ignore the rest? 
Yes to all the above. Everything is possible. 


But not all roads are equally likely to result in success. 
We enjoy stories about outliers because exceptions are in- 
teresting. But exceptions are not repeatable, and there- 
fore they may not be teaching you anything useful. I be- 
lieve the roadmap above is repeatable. Not a guarantee 


of success, but a process that makes sense. 
It’s your life; you can take any road you please. 


What are you going to do? 


Many thanks to Gordon Daugherty, Rowan Udell, and Sathyanand S for 
contributing insights to early drafts. 


Printed from: A Smart Bear 
https://longform.asmartbear.com/product-market-fit-formula/ 
© 2007-2024 Jason Cohen WW @asmartbear 
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Selling to Carol: Why targeting an ICP brings 10x more customers 


than you expected 


by Jason Cohen on January 7, 2024 


Targeting your “Ideal Customer Profile” (ICP) is the best way 
to differentiate and win sales, but does it limit your target 
market? 


"It's not that I don't like you, Ted, you just don't 
fit my target demographic." 


Selling to everyone means selling to no 
one 


Everyone goes to your home page, therefore your home 
page has to speak to everyone, right? Buyers, users, exist- 
ing customers, curious explorers, all working at compa- 


nies large and small. 


Most companies approach this the wrong way, which is to 
speak to “everyone.” The worry is, if you were to speak 
only to a narrow customer segment, everyone else feels 
excluded or confused, and will bounce off the home page 
and buy something else. So you end up with generic posi- 


tioning statements like “The Power To Know.” 


SAS the power to know 


Ssas. 


Giving you The Power to Know® Through innovative software 
and services, SAS empowers and inspires customers around the 
world to transform data into intelligence. SAS is a trusted 
analytics powerhouse for organizations seeking immediate 
value from their data. A deep bench of analytics solutions and 
broad industry knowledge keep our customers coming back 
and feeling confident. With SAS®, you can discover insights 
from your data and make sense of it all. Identify what's 
working and fix what isn’t. Make more intelligent decisions. 
And drive relevant change. 


SAS is the The Power To Know®, but know what exactly? 


SAS has trademarked “The Power To Know®” but I’m not 
sure why. Can you tell me what the product does? Who 


it’s for? Did you read that entire paragraph? Surely not. 


Could anyone in the world—even their perfect, ideal, 
best customer (ICP: Ideal Customer Profile)—be in- 
trigued and excited to learn more? If that ICP happened 
across the website, would they even know they are the 


ideal customer? 


The first step in disabusing yourself of this fluffy lan- 
guage is to see that by not speaking to a specific cus- 
tomer, you’re saying nothing to everyone and thus every- 
one will bounce off the home page, and never think of 
you again. (You didn’t give them anything to think 
about.) If your ideal customer doesn’t know you're selling 


to them, who will? 


By the way, The Penny Group also delivers “The Power To 
Know®,” also trademarked. So much for trademarked 
differentiation. 
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\.: The Penny Group 


The Power to Know’ 


So much power! So much knowing! 


Generalized messaging has no power, no emotional con- 


nection, no interest, no information. 


Exactly the opposite of what you need from your adver- 


tising or home page. 


The next step in disabusing yourself of this idea of 
“speaking to everyone” is to acknowledge that it is im- 
possible to speak to everybody at once. You want soft- 
ware developers to know you have an extensible API, but 
you want the corporate buyers to know that your product 
makes them more money. You want to look professional 
so that managers at large companies are comfortable 
trusting your company, but you don’t want to so aloof 
that small company buyers see you as “too corporate” 
and can’t relate. You want to highlight configurable 
workflows and reports that allow large customers to ap- 
ply your tool to all their teams, but you want small cus- 
tomers to realize that you can turn all that off so things 


are simple and easy. 


Once you agree with these two points—you must be 
evocative and specific, and you can’t speak to everyone— 
you must conclude that you have to speak to just one 


type of customer. 


But how do you pick, and how you avoid excluding 
everyone else, reducing your target market to a tiny 


niche? 


We'll start with the first question. 


Selling to Carol 


Describe a perfect customer. We'll call her Carol. Pick a 
concrete company that she works for, a company similar 
to one of your existing, thrilled customers. What’s her of- 
ficial title and what does she do? If your potential market 


includes a wide variety of company types and positions, 


just pick one in particular. Whatever problems your prod- 
uct solves, Carol has all those problems. She has all the 
problems, she knows she has the problems, she already 
has the budget to spend on those problems, and she’s al- 
ready looking for how to spend it, just like it says in your 
market_analysis. Write those problems down from her 
point of view, the way she would describe them if com- 
plaining to a friend over lunch. Whatever advantages you 
have over your competitors’ Carol needs exactly those 
things. Whatever makes people love you~ Carol has 
those attributes. Whatever makes people continue paying 
for the product forever® Carol has that. List all these 


things* 


Carol is literally custom-built to be blown away by your 


product. 


1 Not sure? Ask your Sales team—they’ll tell you, “oh yeah, any 
time someone says , | know we're going to close the 
sale.“ 


2 Not sure? Ask your Support team—they'll tell you, “oh yeah, 
the best people are because they ask great questions 
and they’re always telling me how much they love us, even 
when they have a problem.“ 


3 Not sure? Look at your retention data—which customers stay 
forever, even if you think they shouldn’t? Not sure about that 
either? Use the Iterative-Hypothesis method to interview them 
—you need to do that anyway so you can build great products. 


4 Some pundits recommend inventing a personal back-story like 
“Carol has two cats and loves Mahler.” Don’t do that. It’s irrele- 
vant and inaccurate. Stick with professional attributes that de- 
termine who Carol is, what Carol is interested in, why she is in- 
terested in it, why she is looking for a new solution right now, 
how she makes those decisions, what her budget is, and so on, 
all topics listed in the aforementioned Iterative-Hypothesis 
method. 


Now the question is: What would a web page / Google 
ad / print ad / trade-show booth / postcard be like, such 
that Carol would immediately understand that you are 
her savior? Remember, you get only 3 seconds to grab 
her attention and another 5-10 to convince her that your 


product is the second coming. 


Can you make it clear in a picture? Maybe a before/after 
she can relate to? Will describing three features make it 
plain? Will pointing out your best competitive advantage 


make her weep for joy in finally finding a company who 
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“gets it?” Can you ask a provocative question, something 


she identifies with? Is there a phrase where she’d laugh 
out loud because “that’s so true?” OUR 
BLADES 
You only get a few seconds, so a paragraph won’t do. You Ex KIN G 
have to communicate in a picture and a few words. The 
. GREAT 


good news is you have to please only Carol, and you 
know Carol. You even know she'll honestly be thrilled to 
find you. 


If your ad can’t grab Carol’s attention—your perfect 
Dollar Shave Club was unabashedly “cheap but good,” and 


customer—why do you think it will grab anyone else’s didn’t care about sleek advertising like Gillette. It got every- 
one’s attention because it was such a clear message, and sold 


attention? : 
razors to everyone, not cost-conscious consumers only. 


If you still say it’s impossible to communicate your mes- 

sage in 5-10 seconds, no one in the world will get your 

message. Simplicity is as important in positioning as it is Cheap a nd “Uely 

in strategy generally. does 1t gAPPy 


When you're willing to speak only to your target audi- 


ence, it is incredibly powerful: 


‘ \ 
= | 
uw / Se 


ASK YOUR DEVELOPER. 


SUBARU 360 /*1297°";. 66 Miles per gallon 


With a face that only a Mother could love, Subaru has__the only 5 door Van, (domestic or import) available 
done it, proven that America is ready fora Cheap & on the American Market, Check the price and take a 
Ugly Little Car. With a Mini-Price, $1297. POE®.and good look. Subaru, still Cheap & Ugly & Making more 
Maxi-Mileage — up to 66 MILES PER GALLON. 

Subaru now introduces 3 great additions. The 
Subaru Sport, the Subaru Truck, and the Subaru Van, 


J af 


SUBARU VAN / $1397 9.02 SUBARU SPORT / $1297 poe SUBARU TRUCK / $1325 poe 
*PLUS $69 SPORT PAK 


SUBARU MAKES SENSE 


SUBARU OF AMERICA 
EASTERN DIVISION® 555 CITY LINE AVENUE * BALA-CYNWYD, PA. 19004 


Twilio targets developers, even though the buyer isn’t a devel- WESTERN DIVISION + 1000 WEST COAST HIGHWAY * NEWPORT BEACH, CALIF. 92660 
oper. This campaign was so effective, it turned into an entire 
book. 


Subaru knows it isn’t cool; it highlights its positive attributes. 
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IT LOVES CAMPING, DOGS AND 
LONG-TERM COMMITMENT. 


TOO BANS NEY A GAR. 


? 


Subaru is about accessible, safe family life. 


Calling it 
transportation is 
like calling sex 
reproduction. 


esonbe it, you risk making the understaternent of the century. The Cayman G14 
is is highest form, It is the surn of all cur knowledge engineered into a machine 
that can take the human senses to the redline. Porsche. There is no substitute. 


The Cayman GT4. 


Porsche’s message is... differentiated from Subaru’s. 


Surprise: You’ll automatically sell far 
beyond Carol 


Now let’s tackle the fear: That targeting Carol means 
you're excluding the rest of the market, and therefore 
limiting your growth and reach, even if you perform well 


in your niche. 


That’s not what happens. Here’s what actually happens: 


You have a target market—the bullseye—and the center 


of that bullseye is Carol: 


Every product has strengths and weaknesses. For Subaru: 


Strengths Weaknesses 
¢ Low-cost ¢ Low performance 
* Reliable * Not sporty 
* Safe (top-rated by government) * Not cool 
¢ Rugged (take it off-road) * Ugly 


Carol is constructed such that she values everything on 
the left, and actively does not want anything on the right. 
To her, those aren’t weaknesses, those are actually 
strengths, because she wouldn’t be caught dead in a 
Porsche, her lifestyle isn’t sporty, and she disagrees with 


your definition of “cool” and “ugly.” 


STRENGTHS WEAKNESSES 
*LOW COST / + LOW) PERFORMANGE- 
*RELIABLE Y = <_ wht | > 2NOF-SPORTY 
“SAFE / ao want @ “NOT COOL 
*RUGGED / *U6EY 
dingee 7 


But Carol isn’t the only person attracted to this set of 
trade-offs. There are other people who either like the 
things on the left or are at least indifferent to them, and 
are indifferent to the things on the right. For example: I 
drive my daughter to school, drive to work, and spend a 
lot of time on highways and residential roads, so I want 
something safe and reliable, and I don’t care about high 


performance or being sporty. 


That’s the next ring in the bullseye—people who general- 
ly value your positives and aren’t dissuaded by the weak- 
nesses. There are 10x more of those people than there 
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are of Carol. Let’s call that person Diana: 


STRENGTHS WEAKNESSES 
“Low cost \/ * LO\ PERFORMANCE 
DIANA 
RELIABLE. \~ : ; ees 
“SAFE. \/ CAROL (donk eae, “NOFCOOL 
BUGGED sub 

(dont care) 


But that’s not all. Buying a car—or anything—is always a 
set of trade-offs. You rarely find a product that fits your 
preferences perfectly, i.e. you’re rarely The Carol for any 
product. So, you weigh the pros and cons. Maybe I don’t 
care about being rugged because I don’t go off-road, but I 
do care about safety and reliability, and ’m neutral on 
cost. And on the weaknesses, I’m disappointed that it’s 
low-performance, and I do care what it looks like. But, I 


don’t care about “sporty” or “cool” specifically. 


So where does that leave me? Who knows—everyone 
will weigh these things differently. Clearly some of those 
people will still consider the Subaru to be the best-possi- 
ble set of trade-offs, and there are 10x more of those 
kinds of people than there are of Diana. Call them 
Eddie: 


STRENGTHS WEAKNESSES 
N 

-Lowcost— (Low PERFORMANCE) 

“RELIABLE »/ -NOT-SPORFY 

“SAFE \/ 2NOF-6OOL- 

LRUGGED- 


This is why targeting Carol means your target market is 
at least 10x larger (because of Diana) and more like 20x- 
100x larger (because of Eddie). 


Targeting Carol makes your message clear, compelling, 
evocative, even emotional. But trade-offs are how people 


buy, and exactly because you were so clear about what 


those are, you’ve paved the road for many people to 


make that choice. 


Does this theory work in practice? 


Continuing the Subaru example’, the advertisements 
above are about being low-cost but practical. These are 
attributes valued by many people. But in the 1990s, sales 
were in decline, so they researched which consumer seg- 
ments were most likely to buy a Subaru. They found five 
categories: Teachers, healthcare professionals, IT profes- 
sionals, “outdoorsy types,” and lesbians. Lesbians were 


the strongest category—four times more likely to buy. 


> This story is expanded and cross-referenced in Alex Mayyasi’s 
2016 article in Pricenomics. 


Targeting lesbians was a risky move in the ’90s. There 
were still no mainstream TV shows or movies with gay 
characters, and celebrities were still in the closet. When 
Ellen DeGeneres (both a celebrity and star of a popular 
TV show) came out as gay in 1997, advertisers pulled 
out, including Chrysler. When Ikea ran a commercial fea- 


turing a gay couple, their stores received bomb threats. 


But as Warren Buffett says, “Be greedy when others are 
fearful,” and Subaru invested where others feared to 


tread. 


Entirely. comfortable 
with itssorientation. 


The New Subaruekorester 
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It's Not a Choice. 
It’s the Way We're Built. 


Subaru All-Wheel Driving System. 
In every car we make. 


Twenty years before Lady Gaga’s “Born this way.” 


The campaigns were a hit, but not only with lesbians. 
Subaru grew faster in the subsequent ten years than any 
other car company® and while the gay and lesbian demo- 
graphic remained their strongest niche, the vast majority 
of people who bought a Subaru were straight. And they 
were buying the car because they value its attributes—af- 
fordable, safe, rugged—and because on balance they 
cared more about those than the attributes it lacks— 


sleek, sporty, high performance. 


When you target Carol, your positioning is strongest, and 
you win where you deserve to win. But far more people 
will buy because they have similar purchasing 


preferences. 


6 Measured as a percentage of total sales volume from 1995 to 
2005, of car companies that were founded before 1990. 


Finding Carol in the noise 


DZ 
@DanielZarick 


Two weeks ago an @ArrowsHQ prospect offered to pay us 
$48,000 same day § He was eager to start and would 
bring tons of usage—and yet, we turned him down. 


The decision took <5 minutes, and we were confident it 
was the right call. 


But why? 


When a customer cancels, we need find out why—what 
did they think would happen, and how did we fail them? 
Unless the customer stopped their project or went out of 
business, cancellation indicates a gap between what we 
promised them (marketing) and what we delivered 


(product). 


This investigation is most urgent when the customer is 
Carol. You stipulated she is the perfect customer, but she 
disagrees. Perhaps the product needs to change, but per- 
haps your definition of Carol needs to change to better 
match what the product has turned out to be or what you 
are uniquely excellent at delivering, or some other facet 


of your strategy. 


The same could be said Diana, but when it comes to 
Eddie it’s less clear whether this is signal or noise. Eddie 
is a mis-match for some of our attributes, and might can- 
cel as a result, but that doesn’t mean we should change 
those attributes, if they align with Carol and Diana. If we 
treat all customers as if they are Carol, and therefore all 
cancellation data as a vital signal, we lose track of build- 


ing for Carol and we end up building for nobody. 


Some customers aren’t even Eddie—they’re not on the 
bullseye, yet they signed up. You might think this would 
be rare, since it is irrational, but in fact it will happen all 
the time. You need to identify these customers and com- 
pletely ignore their data—the noise will push you off 


course. 
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Therefore, you need a way of identifying who is a Carol. 
Ideally, right from the start, during on-boarding, maybe 
with a few questions, maybe based on activity indicating 
they’re in that perfect zone. Then, you can measure can- 
cellation rates of Carol (the only rate you care about), 
and spend extra time finding out why they cancel. The 
“extra time” comes from the time you saved by not pur- 
suing and interviewing other customers. 


ANDERSON 
“All I’m saying is just once I’d 
like to be sitting without being 
a target.” 


Of course it’s not just in cancellations that you want to 
segment the ICPs from the rest of the customers. It’s in 
your activation funnel. It’s in your trial or freemium con- 
version rate to paid. It’s in individual feature usage. It’s in 
conversion to monthly active users (meaning a single 
user actually using it consistently month over month). It’s 
in tech-support requests both for bug fixes and features. 
It’s in every aspect of understanding how customers are 
intersecting with our product and company. It’s always 
about the Carols; everything else is noise, and if you 
don’t separate the two, you can’t build the best product, 


for any definition of “best.” 


The typical failure mode for this is that your biggest cus- 
tomer, who you desperately want to keep because you re- 
ally want that revenue, constantly occupies you in tech 
support and in feature requests, some of which you’ve 
implemented just for them, with the internal excuse that 
“Sf they want it, other larger customers probably will 


want it, so this is a path of growth,” yet they’re still 


perennially unhappy, unsatisfied, obviously a bad fit, an 
anti-Carol, but dangit you need that money and if we can 
just make this work there must be more customers like 


that coming. 


And after all that, of course they leave in a huff, having 
been been incredibly unprofitable despite their high 
nominal revenue, and you should turn around and say, 
“Wait a minute, that customer was not in our bullseye all 
along.” Rather than noticing that from the start, and 
helping them to leave your company, you clung to them 
because you so desperately wanted it to work, and for 
good reason—it was a lot of money—but ultimately that 
wasn't the right reason, and the whole thing was an 
enormous waste of time. You could have been finding 
and on-boarding a few more Carols instead, and they 
would still be here, happily paying you money, profitably. 


You'll know that you’re doing this right when you turn 
down a sale, even though it would have been your 
largest customer, simply because they're not your ICP as 
in these miniature stories from Daniel Zarick at 
ArrowsHQ and John Doherty at EditorNinja. This sepa- 
rates first-time founders from seasoned founders. It’s 
not a coincidence that both Daniel and John have seen 
this movie before, made this mistake before, and are 
avoiding it this time. 


Can you? 


After all this, if you reread that ad from SAS with The 
Power To Know®, you'll die a little inside. So much noth- 
ing. Nothing to say, said to no one, caring about nothing, 
changing nothing, being nothing. 


You can do better. 


And make more money doing it. 


Many thanks to Florian Caeser for contributing insights to early drafts. 


Printed from: A Smart Bear 
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The “errors” that mean you’re doing it right 


by Jason Cohen on January 21, 2024 


Some things appear to be mistakes, but in fact should be 
celebrated as the expected outcomes of great decisions. 


If you don’t make mistakes, you’re 
not working on hard enough 
problems.” 

—Frank Wilczek, 2004 winner of Nobel 
for Physics 


Intellectually we know that failures are inevitable when 
we're striving, growing, and learning. In practice we’re 
not always so understanding when it comes to our teams, 
our revenue targets, and especially when it comes to 


flogging ourselves. 


Indeed, not all errors signify progress. Some are negli- 
gence, or just bad luck. We don’t always learn from fail- 
ure; in fact, sometimes there’s nothing in particular to 


learn. 


ami 
(ni ALY =, 


“OK, lesson learned. Going forward no one listen to Don.” 


The following “errors” are the natural by-product of good 
decisions, or the result of a fundamentally positive cir- 
cumstance that is attended by the proverbial “good prob- 
lem to have.” Most demand a response, but they should 
be regarded as a necessary side-effect of success, and cel- 


ebrated as such. 


Re-adding features/bugs you removed 
from the backlog 


If you’re not adding back feature-requests or bugs you 
cleaned out of the backlog, you’re not cleaning out 


enough. 


Backlogs grow without bound unless they are culled. 
1000 tickets is the same as 100 tickets, except that you 
haven't identified which 10% are most important. Which 
means you're definitely not working on the most impor- 
tant ones. But if you delete things, it will sometimes turn 
out we needed to do it after all. That’s a sign that you’re 
handling your backlog well. (In part for this reason, you 
should have multiple backlogs, except for the work 


youre doing right now.) 


Pivoting a strategy just after creating it 


A strategy that never changes is wrong, and the most 
likely time to discover that it’s wrong is just after you 
wrote it down, because you have the least practical expe- 
rience with how it intersects with real life. 


If you’re not pivoting a strategy that turned out to be 
wrong, you’re penalizing the company with months or 
even years of useless work, followed by rework (Gif you 
haven’t run the company into the ground), putting the 


entire future of the company at risk. 
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“What's that boy?! A paradigm shift?!” 


Great strategies are hard to create, and released with 
great fanfare: Sparkling documents, inspiring presenta- 
tions, pulpit-thumping speeches, reorganized teams and 
strategic-pillarized work. So the last thing you want, is 
come back in a month and say, “just kidding, we were 
wrong about something important.” 


Will you lose credibility? Will anyone believe the new 
strategy? Will people think “management doesn’t know 
what it’s doing?” These are risks you have to take, be- 
cause executing the wrong strategy is far worse. Indeed, 
this is the expected result of a new strategy; it’s highly 
unlikely you got everything right the first time. The best 
way to communicate, is to say everything in this para- 
graph out loud, so everyone knows that you know that 
they know, and that you’re putting the company first, and 
ensuring that no one is doing work that we secretly know 


is the wrong work. 


Refactoring infrastructure after growing 
10x 


If you’re not refactoring your infrastructure after a ten- 
fold increase in growth, you over-engineered your origi- 


nal infrastructure. 


Having scaled WP Engine to millions of websites serving 
tens of billions of requests daily, I can tell you that scale 
is hard, and that you don’t know what will break under 
scale until you’re already scaling (because you have to 


handle things that you can’t even measure yet). If you 


over-engineer your original product, you’re simply not 
shipping your SLC fast enough, and your naive attempt 
at engineering massive scale from the start is just another 


form of premature optimization. 


Adding words because messaging was 
too terse 


If you’re not adding back words because the messaging 
was so terse that it became confusing, your marketing is 
too verbose, too fluffy, and probably doesn’t know what 


it’s trying to say. 


We are an innovative, 
forward-thinking 

organisation that make 
cutting-edge 

telec Oommunication 
solutions and industry 
leading 


equipment. 


Get to the point. 


® STAEDTLER 


People don’t read. Tweets are short. Google Ads are 
shorter. Email titles are shorter. People bounce off home 
pages in three seconds. No one reads the paragraph of 
text in the dialog box. It’s 100x more likely that your 
messages aren’t punchy enough, aren’t specific enough, 
than that theyre so brief as to be unintelligible. 
Nowadays you can use ChatGPT prompts to get you 80% 


of the way there, so you have no excuse. 


2 of 4 


The “errors” that mean you’re doing it right 


Adding back features you removed 


If youre not adding back features you removed, you’re 


not removing enough. 


Many products never remove features. This indicates 
we're not being critical enough, not weeding our garden, 
not learning what customers really want, not under- 
standing what’s useful, not admitting when we got it 
wrong, not shifting when the market shifts. When we do 
remove a feature, sometimes it will turn out the feature 
really was important after all. While of course in a per- 
fect world we wouldn’t have made that mistake, it’s a 


natural consequence of weeding. 


Fixing lots of bugs just after a major 
release 


If you're not fixing bugs due to releasing quickly, you re- 


leased too late. 


ANveesan 


“Could you hurry it up, sweetheart? We're going 
to be late!” 


wait until “everything’s perfect.” Windows 95 shipped 
with tens of thousands of known bugs}, and was herald- 
ed as one of the greatest software innovations and most 


successful product releases of its time. Contact with cus- 


tomers lets you know which bugs are more important to 
fix next, and always reveal new bugs that you weren’t go- 


ing to find on your own anyway. 


1 There were so many, there’s an entire book explaining how to 
work around 1000 of them. 


Waiting too long to scale support or sales 


If you held onto support and sales for too long, rather 
than hiring a team, you learned a lot about your cus- 


tomers, and a lot about how to do Support and Sales. 


It’s a classic funded-startup mistake to scale out either of 
these organizations too soon. Without a system in place, 
with materials, knowledge bases, and scripts, new hires 
don’t know how to do the job. A distributed-work envi- 
ronment makes this 10x more challenging. The second 
you put someone else between you and a customer, your 
pace of learning and understanding falls off a cliff. Wait 


until it’s breaking for lack of scale~, then scale. 


2 Once at scale, this rule no longer applies; at that point, you're 
mismanaging the company 


Letting someone go soon after hiring 


If you held onto someone even though you knew isn’t 
was never going to work, you’re doing a great disservice 
to that person, and your team, and your company, and 
yourself. 


Of course this shouldn’t be done capriciously, but no one 
benefits from dragging it out. One likely outcome is that 
you lose good people, because they see you building a 
team they don’t want to be a part of. Another is a deluge 
of meetings, complaints, side-conversations, and general 
worsening of morale. And lower productivity, as compe- 
tent people cover for the incompetent. You have to face 
the truth and act quickly. If this is happening a lot, it’s 
also urgent that you fix your hiring process; in the mean- 


time, the rule still applies. 


3 of 4 


The “errors” that mean you’re doing it right 


Ignoring a competitor’s move that turned 
out to be important 


If you’re not ignoring most of your competitor’s moves, 


then you’re playing their game, not yours. 


"AS you can see, we've not only kept up with the 
Jones, but surpassed them. Next up, the Nelsons." 


It’s essential to stay focused on your unique value propo- 
sition and not get sidetracked by every move your com- 
petitors make. For instance, when a major competitor of 
Dropbox launched a similar service at a lower price, 
Dropbox chose to stay the course, focusing on their supe- 
rior user experience and brand loyalty, rather than en- 


gaging in a price war. 


Sometimes it will turn out that you really do need to re- 
act, but that signal will come from customers, in the form 
of them asking for features “because so-and-so has it” or 
cancelling and going to a specific competitor. That indeed 
demands a reaction, but only because you're seeking 
what’s genuinely best for your customers, not because 


you're reacting to everything that competitors do. 


Rejecting a lucrative, distracting deal 


If you’re not rejecting lucrative deals that don’t align with 
your strategy, then you don’t have a strategy. If you’re not 
rejecting relationships that don’t align with your core val- 
ues, you don’t have core values. 


Money is too tempting to reject. Money is one of the 
main reasons you’re building a company in the first 
place. Money is what keeps the doors from closing and 
enables the next set of things you want to do. It’s even 
wise to say “yes” instead of “no,” so long as there’s 
enough money in it. 


But money is not more important than strategy, and it 
cannot be more important than your values, otherwise 
youre saying that you don’t actually have either one. 
There’s always a way to make more money—a different 
product, different industry, or breaking the laws or being 
unethical. There’s a reason why you're taking the path 


you're currently on. 


Not all problems are bad. It’s easy to be hard on our- 


selves, but sometimes we should do just the opposite: 


Celebrate our devotion to good decisions and good strat- 


egy, even when it has negative consequences. 


That means you have a strategy, and have the ability to 


make the tough, wise decisions. 


HT Hassy Veldstra for finding the Staedtler advertisement. 


Printed from: A Smart Bear 
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Deciding whether an investment is worthwhile 


by Jason Cohen on February 4, 2024 


Why “expected value” doesn’t work; here’s a better 
framework for making long-term investments in your career, 
startup, and life. 


NAS es ee 


“AnderCorp surprised investors expecting the 
worst today by performing completely 
miserably.” 


Whether it’s your career path, which job to take for the 
next four years, investing in your personal brand, creat- 


ing a startup, or investing in a startup, the question is: 
How do I decide which investment is worthwhile? 


I’m defining an “investment” as distinct from other bets 


by having these characteristics: 


1. Spending time and money and reputation for months 
or years, hoping for a large return in the far-future. 

2. Can do only 1-2 of these simultaneously, because you 
are “all in.” 

3. Because the future is uncertain, there is significant 
risk: Whether it will pay off at all, and if so, by how 


much. 


This is in contrast to short-lived experiments, like a bet in 
poker, an A/B test, or making a brochure for a product 


concept to see whether people will buy it. Bets, experi- 


ments, and trials are excellent techniques, but by “invest- 
ment” I mean a big commitment (time, money, reputa- 
tion, opportunity cost), over a long time-frame, with a 


large potential payoff, under conditions of uncertainty. 


Tackling this combination of constraints leads to a frame- 
work for evaluating which investments are worthwhile. 


Don’t use “Expected Value” 


Economics instructs us to use “expected value” to evalu- 
ate investments. This is almost always the wrong tool for 


the job. 


The theory appears sound: “Expected value” means mul- 
tiplying the value of the outcome by the probability of at- 
taining that outcome, resulting in a single number that 
encompasses both risk and value; a “risk-adjusted value,” 


as they say in finance. 


Let’s take an example to see why this number isn’t useful 


for evaluating investments. 


Investment “A” requires you to spend $50,000, and takes 
an entire year of your life. The payoff is $1,000,000. 
Furthermore, the probability of payoff is 100%—it’s a 
signed contract with cash in an escrow account. 
Therefore, the “expected value” is $1m/ and the return 


on our investment is 20x” 


Great investment! Now consider investment “B”—you 
also spend $50,000 and a year of your life, and this time 
the payoff is a whopping $10M, but the probability of 
that payoff is only 10%. The other 90% of the time, you 
receive no money at all. The “expected value” is also 
$1M>, the same as investment A. And therefore invest- 


ment B is also a 20x payoff. 


1 Because 100% x $1M = $1M 


2 Because the $1M expected value is 20 times the cost of $50k. 
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3 Because (10% x $10M) + (90% x $0) = SIM 


Perhaps you already see the flaw in the logic. With in- 
vestment B we will almost certainly lose money and lose 
a year of our life, yet we’re still claiming it represents a 
“20x payoff.” We’re also claiming that B is economically 


indistinguishable from A. 


But unless you already have many millions of dollars in 
the bank, you probably wouldn’t pick investment B, be- 
cause investment A is a guaranteed life-changing amount 
of money, while B is almost certainly a waste of a year 


and $50k. They are not even remotely equivalent. 


It’s even worse in real life because you rarely know either 
number—neither the risk nor the payoff. Whether it’s the 
stock market, a startup, or whether a certain career path 
will still be lucrative in ten years, even experts can’t pre- 
dict probability or outcome. So, what garbage numbers 
are you multiplying together to get “expected value,” and 


isn’t the result even worse garbage? 


There is one case where expected value is the right tool: 
A portfolio of bets. When a VC invests in twenty compa- 
nies, when you write 100 newsletters and 1000 tweets, 
when you play 1000 hands of poker, when you have a 
balanced, uncorrelated financial portfolio, then you'll 
have some winners and some losers, and they roughly 
balance out, and therefore an “average” like expected 
value is indeed what you should expect. This relative sta- 
bility in outcome is why a portfolio of asymmetric bets is 
a good strategy. The types of investments covered in this 
article aren’t like that—e.g. what job to take next, what 
startup you're going to spend your savings on—but if you 
can place a number of uncorrelated‘ bets, then expected 


value becomes useful. 


4 Correlation is a key factor which often brings down portfolios, 
including angel, VC, and financial. “Correlation” means “as 
goes one, so very likely goes the other.” In that case, you’re 
back to the original scenario. Only if they are independent— 
like successive coin flips—can you employ “the average” to de- 
termine the likely total result. 


Mandate an outsized payoff 


In the success case, the investment must have an enor- 


mous payoff. 


If the payoff isn’t huge, then it’s not worth the 
(unknown!) risk. Not just the risk that it pays off at all, 
but the magnitude of the payoff. When it can pay off 10x 
or 100x, or when it can be life-altering, then even if you 
fall short of the goal—only 3x or only life-improving— 


then it was still worthwhile. 


If riskier investments could be 

counted on to produce higher 

returns, they wouldn’t be riskier.” 
—Howard Marks 


How do you define “enormous?” How can you calculate 


the payoff, especially with things like career paths? 


Use Fermi estimation. As I also suggest for ROI calcula- 
tions, probabilities, and evaluating markets, estimate to 
the nearest power of ten. Estimation in detail is difficult 
and often incorrect; estimating to the nearest power of 


ten is easy, often requiring little research. 


For example, if one career path means earning 50% more 
per year for ten years (after you get established), that’s 
10x more earning power over your career. Or a startup 
targeting a small, stagnate niche might someday be 
worth $1M, whereas a small software company in a larg- 


er, growing space could be worth $10M. 


You'll be tempted to round up; in fact I just did in the 
“50% more earning power” example. Don’t do that, be- 
cause you're exaggerating how good the investment is; 
typically if there’s any error, it’s in the direction of being 
less valuable. If, for example, your main career goal is to 
maximize your earning power, then “50% more” is not 
good enough; you need to find a different route that po- 


tentially doubles your current trajectory”. 
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5 2x more for 10 years is actually 20x of course, but now round- 
ing down to 10x makes sense, especially since it might not be 
2x exactly and it might take a few years to ramp up. 


This is the same logic as selecting Rocks in the Rocks, 
Pebbles, Sand _ prioritization framework. Your large per- 
sonal investments are Rocks, and must be evaluated 


accordingly. 


Accumulate value over time 


For the payoff to be large, value needs to accumulate, 
which means it needs to be possible to add value reliably, 
and for value-already-added to not erode quickly. At a 
minimum, value must accrete, but ideally you construct a 
positive feedback loop, where growth or size itself begets 


more growth. 


Social media and SEO are like this—the more you write, 
the more others interact with you, the more others link 
to you, the more attention you get, the more traffic you 
get, which makes it easier to get more attention and in- 
teractions. Writing consistently increases your ability to 
write well and often, and thus there is a positive feed- 


back loop of growth and value. 


Product companies are like this, especially with the re- 
curring-revenue business model, where the revenue 
grows so long as each months there are more customers 
starting o use the product as there are stopping, which 
will happen so long as you’re making a product that peo- 
ple truly want and love. (Here’s my roadmap _ to 
Product/Market Fit for how to build such a company.) 


Consulting and professional services companies are often 
not like this. You'll have regular customers, but the rea- 
son consulting companies and accounting and law firms 
are often valued around 1x revenue or less, while SaaS 
companies with similar growth-rate are valued at 5x — 
10x revenue, is because the work you do for a client isn’t 
accumulating and applying automatically to all other 


clients, and consulting companies are hard to scale. 


The most powerful force in the 
universe is compound interest.” 
—Albert Einstein. 


Some people insist that investments must be exponential, 


but often they don’t understand what that means, believ- 


6. 


ing for example that “viral products” are exponential, 


when in fact they are not. Exponential is wonderful of 
course, but not required, and in fact is probably a red 


herring for most ventures. 


Stable success conditions over ten years 


Just because the world values something today, doesn’t 
mean it will be valued ten years from now. If you spend 
3-5 years investing, expecting a payoff in the subsequent 
5 years, the world had better still value that thing during 
years 5-10. 


This summarizes Jeff Bezos’s famous explanation of why 
Amazon’s strategy has always been based on “what will 
not change.” He articulates it so well, I'll leave it here 
without further commentary (except my emphasis, in 
bold): 
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I very frequently get the question: “What’s going to 
change in the next 10 years?” And that is a very inter- 
esting question; it’s a very common one. I almost never 
get the question: “What’s not going to change in the 
next 10 years?” And I submit to you that that second 
question is actually the more important of the two—be- 
cause you can build a business strategy around the 
things that are stable in time. ... [I]n our retail busi- 
ness, we know that customers want low prices, and I 
know that’s going to be true ten years from now. They 
want fast delivery; they want vast selection. 


It’s impossible to imagine a future ten years from now 
where a customer comes up and says, ‘eff, I love 
Amazon; I just wish the prices were a little higher.” “I 
love Amazon; I just wish you’d deliver a little more 
slowly.” Impossible. 


And so the effort we put into those things, spinning 
those things up, we know the energy we put into it to- 
day will still be paying off dividends for our customers 
10 years from now. When you have something that 
you know is true, even over the long term, you can 
afford to put a lot of energy into it. 


—dJeff Bezos, 2012 (it’s been 10 years since this quote, so 
we can confirm this was indeed the correct analysis.) 


WHACK-A-MOLE SALES 


“Second quarter is up. No, fourth quarter. 
Now second again. OK, it’s third now...” 


Align with global trends 


If the best investment is in something that doesn’t 
change, the next best is aligned with a global trend with 
sufficient momentum that it is unlikely to change soon, if 


ever. 


For a startup, you want to be in a large, growing market. 
This maximizes the number of niches you might find 
traction in, without triggering a reaction from a large in- 
cumbent. It maximizes the number of marketing chan- 
nels you might be able to leverage. It means people are 
already spending money, so you’re answering “why buy 
from me” rather than “why should I allocate budget for 
this at all” or arguing against “that’s a problem but not a 
priority” or evangelizing against “I never heard of that 
problem.” It means the pie is growing, even if your per- 


centage of the pie is small. 


Other current trends are the increase in virtual and dis- 
tributed workforces, the increasing average age of hu- 
mans on Earth, the increase in physical delivery services 
and online shopping, the increasing global demand for 
health care, education, and energy, and of course the rise 
of AI in unlocking novel features and capabilities for 
products and services that are changing the nature of 


work and life. 


The stock market is a device for 

transferring money from the 

impatient to the patient.” 
—wWarren Buffett. 


A trend you can depend on is either (a) so large today 
that it would take a decade to change significantly, or (b) 
is growing so fast (e.g. at least 5x in next 5 years) that it 
will be a strong trend even if your growth estimate is sig- 


nificantly wrong. 


Have an “edge” 


Why should you make this investment, not someone else? 
A good investment, that you cannot execute with excel- 
lence, isn’t a good investment for you. 


What’s your edge? 
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Deciding whether an investment is worthwhile 


In financial or startup investing, the edge is often your 
ability to evaluate the risk. You might understand that a 
certain execution in a certain product in a certain market 
carries lower risk than others believe. This is an edge, 
though it is a rare one, in part because even experts are 


bad at evaluating risk (and the majority of VC funds lose 


money, and Angel investors are even worse). 


More likely it’s something special about your execution. 
I’m avoiding the word “unique,” because although it’s 
great if you’re truly unique in some way, this is rare, and 
in fact unnecessary. You should, however, leverage your 
strengths so that you have an edge, despite also being 


worse in most other areas. 


One way to be special is to be the top 0.1% in the world 
at something (e.g. Olympic athlete), but that’s probably 


not what you are. The other way, and the way available 


to all of us, is to be in the top 25% at a few uncorrelated® 


things, which together creates a special edge. 


The quintessential exponent of this idea is Scott Adams, 
writer of the Dilbert cartoon and now political activist, 
who points out that he is a decent illustrator but not 
great, a decent humorist but worse than any comedian 
on Netflix, and has held jobs in the tech sector, which al- 
most no artist or comedian has done, and therefore it is 
at the intersection of those three “circles of competence” 


that he has something unique to give to the world. 


6 For example, being great at both math and computer program- 
ming doesn’t make you any more special, because those often 
occur together, whereas being artistic and great at program- 
ming places you in a much more elite group. 


“Passion” can be part of an edge, but it is not enough. 
Passion creates motivation, which is especially important 
when you have to push through the difficult times and 
persevere. However, having passion doesn’t mean the risk 
is low, the payoff is high, the market is good, the cus- 
tomers are there, the product is desirable, or the career is 


lucrative. 


Having an edge reduces risk while increasing not only 
the potential size of the payoff, but your own personal 
fulfillment, because it puts you at the center of what you 
love, what you’re good at, and what the investment 


needs to be successful. 


Align with your long-term vision 


If you run quickly in the wrong direction, that’s not 


progress. 


If you make investments that pay off, but the pay-off isn’t 
what will make you proud, or happy, or fulfilled, or rich, 
or whatever your goals are, then it’s not a good 


investment. 


If you don’t have a long-term vision for yourself, consider 
making one. Taylor Pearson has some great advice on 
how to make a vision and how to set short-term goals 
against it. See also the last section of my article on fulfill- 
ment, with a series of questions that help you discover 


what you love and hate. 


In much of my life, I haven’t had a vision. Fortunately, my 
path has largely been in the right direction anyway, but I 


wonder how much of that is due to luck. 


Best not to leave it to luck. 


Good investments are rare 


With all these restrictions and rules, it’s hard to find a 


good investment! 


Given our size, we see few good 
things [to invest in]. If we were 
smaller, then we’d see lots of good 
things.” 

—wWarren Buffet. 


Yup. Most startups fail. Most VC portfolios lose_ money. 


80% of day-traders lose money. 
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Deciding whether an investment is worthwhile 


That’s why it’s so important to adhere to these guidelines 
—because most investments aren’t right for you, and you 
don’t want to spend the next five years doing the wrong 


thing. 


Of course you'll never know if you made the best choice, 
even with hindsight. So the goal isn’t to make the “best 


choice,” but rather to make an excellent choice. To invest 


wisely, with the information you have. 


Good luck! 
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